gmane.comp.python.lxml.devel

Compilation of lxml on Windows 64-bit

Hanni Ali — 2008-10-02T13:56:53

_______________________________________________ lxml-dev mailing list lxml-dev< at >codespeak.net http://codespeak.net/mailman/listinfo/lxml-dev

Warning: .stabs: description field '10543' too big, try a different debug format

Terrence Brannon — 2008-10-02T00:45:18

_______________________________________________ lxml-dev mailing list lxml-dev< at >codespeak.net http://codespeak.net/mailman/listinfo/lxml-dev

Internal compiler error when compiling lxml

Joao Moreira — 2008-09-30T21:37:58

Hi all, I've just installed the latest lxml and tried to compile it (first I had to get the latest libxml2 and libxslt), it crashes on the first file lxml.etree.c with a "internal compiler error". I'm on a Linux box running Fedora 5, with python 2.4.2. Should I report this as a bug somewhere else ? Any suggestions on how to work around this ? maybe a problem with version compatibility, between lxml and python ? or libxml2, libxslt ? this is my very first attempt at installing anything written in python, please forgive my ignorance. Detailed trace below. Thanks for helping, Joao --------------------- [joao< at >fox lxml-2.1.2]$ python setup.py build Building lxml version 2.1.2. NOTE: Trying to build without Cython, pre-generated 'src/lxml/lxml.etree.c' needs to be available. Using build configuration of libxslt 1.1.24 Building against libxml2/libxslt in one of the following directories: /w7/u/libxslt-1.1.24/lib /w7/u/libxml2-2.7.1/lib running build running build_py running build_ext building 'lxml.etree' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fPIC -I/w7/u/libxslt-1.1.24/include -I/w7/u/libxml2-2.7.1/include/libxml2 -I/usr/include/python2.4 -c src/lxml/lxml.etree.c -o build/temp.linux-i686-2.4/src/lxml/lxml.etree.o -w src/lxml/lxml.etree.c: In function $-1òø__pyx_f_4lxml_5etree_17_IterparseContext__setEventFilteròù: src/lxml/lxml.etree.c:73968: internal compiler error: in merge_alias_info, at tree-ssa-copy.c:235 Please submit a full bug report, with preprocessed source if appropriate. See for instructions. Preprocessed source stored into /tmp/cc9Y1mVr.out file, please attach this to your bugreport. error: command 'gcc' failed with exit status 1 [joao< at >fox lxml-2.1.2]$

Writing TargetParser in Cython

Max Ivanov — 2008-09-29T14:56:41

Hi all! I'm trying to write TargetParser in Cython just to compare perfomance. The problem is with data types. If I define data method as "def data(self, char *data):" I'm unable to use it as TargetParser. I get " def data(self, char *data): UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-4: ordinal not in range(128)" error. I could instance and directly call data() and close() methods and everything works fine, but it refuses to work with lxml. Small testcase following: ----- _target.pyx ----------- cdef class Target: cdef list _data def __init__(self): self._data = [] def data(self, char *data): self._data.append(data) def close(self): return ''.join(self._data) ---- end of target.pyx ------ ---- test.py ------- # -*- encoding: utf-8 -*- import lxml.html from lxml import etree from _target import Target res = etree.HTML(u"ABCD", parser=lxml.html.HTMLParser(target = Target())) -------end of target.pyx ------

Simple doctypes not in docinfo.doctype

F Wolff — 2008-09-29T10:49:15

Hallo list I've hit a snag with lxml and a DOCTYPE decleration. I don't know if I'm to blame here, but would appreciate help either way. I've tried this with an old (1.3.2) and newer (2.0.6) lxml version. (this example is roughly based on the code at http://codespeak.net/lxml/tutorial.html) from lxml import etree from StringIO import StringIO tree = etree.parse(StringIO("""""")) tree.docinfo.doctype '' in the wild in Qt .ts files). My real issue is round-trip problems in a reading-writing cycle where the DOCTYPE is lost, but I guess not being able to use .docinfo.doctype is already a problem. Any help will be appreciated. Keep well Friedel -- Recently on my blog: http://translate.org.za/blogs/friedel/en/content/vrot-mango _______________________________________________ lxml-dev mailing list lxml-dev< at >codespeak.net http://codespeak.net/mailman/listinfo/lxml-dev

HTML Meta Content-Type Tag not created as documenationstates?

John Krukoff — 2008-09-25T23:33:11

So, I was trying to figure out what happend to my meta tags when using the lxml.html module, and saw the note in the documentation that html.tostring will handle them as so: However, that doesn't seem to actually be the case. It looks like etree.tostring is never creating the meta tag as html.tostring appears to expect, and instead the include_meta_content_type flag is simply controlling whether any found meta tag is removed from the output (with an re!). Python 2.5.2 (r252:60911, Sep 22 2008, 12:08:38) [GCC 4.1.2 (Gentoo 4.1.2 p1.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. True ) '

' Not present there, so I figure maybe it's because it's not being treated as a complete document? True ) '

' Parsing as document doesn't create it either. = True ) '

\xa92008

' Okay, maybe it's because I'm using the default encoding for HTML (us-ascii)? Nope, trying something else doesn't cause it to exist either. include_meta_content_type = True ) '\n

\xa92008

' Maybe wrapping in an ElementTree? Get a doctype declaration out of that, but still no meta tag. u'2.1.2' In further testing, it appeared that if a Meta Content-Type tag was specified, it was passed though as is, as long as include_meta_content_type was True. The really weird part of this for me though, is that I've set include_meta_content_type on my much more complicated application server, and it does in fact appear to be generating meta tags automatically (or at least something in my XSLT heavy processing chain is). My testing was an attempt to duplicate that, and I was quite surprised when I couldn't. I've tried this on boxes with both libxml2 2.6.26 (RHEL5) & 2.6.32, and didn't see a difference there.

lxml parser encodings? What's supported?

John Krukoff — 2008-09-17T19:08:43

So, I've been trying to deal with some places where I need to force the parser's encoding, and I've been surprised by how little it seems to support. Specifically, 'ascii' isn't a supported encoding: Python 2.5.2 (r252:60911, Jul 31 2008, 15:38:58) [GCC 4.1.2 (Gentoo 4.1.2 p1.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Traceback (most recent call last): File "", line 1, in File "parser.pxi", line 1240, in lxml.etree.XMLParser.__init__ (src/lxml/lxml.etree.c:58722) File "parser.pxi", line 711, in lxml.etree._BaseParser.__init__ (src/lxml/lxml.etree.c:55050) LookupError: unknown encoding: 'ascii' u'2.1.1' I checked the libxml2 documentation, and that claims that on linux it supports all the encodings that iconv does, which is quite a lot. Almost none of those returned by iconv actually work, though. Am I doing something wrong here by trying to specify the encoding in this way? Is there something weird about my build? If everything is working as intended, is there anyplace I can find a list of the encodings lxml does support? My current workaround is to do the decoding to unicode first, then hand the unicode string to lxml, but that seems less efficient than letting the parser handle it.

Build error of lxml 2.1

Owen Zhang — 2008-09-16T20:43:00

_______________________________________________ lxml-dev mailing list lxml-dev< at >codespeak.net http://codespeak.net/mailman/listinfo/lxml-dev

Best practice for lxml and html5lib

Fabien — 2008-09-16T09:42:26

Hello, I'm trying to make lxml and html5lib working together and I must say that I've some difficulties to find the best solution. lxml 2.2 seems to works well with lxml.html.html5parser but I'm a bit reluctant to put alpha software in production. And when using lxml 2.1.x and html5lib "lxml" tree builder, I get "ValueError: Invalid attribute name u'xml:lang'". Any help for the best combination of lxml and html5lib is welcomed.

resolvers

Steven Vereecken — 2008-09-15T21:43:50

Hi, I 've got a couple of strange things with custom resolvers: 1) I can't get the resolve_file method to work, not with actual file objects, and not with StringIO either. I always get the following message: Exception exceptions.TypeError: 'function takes exactly 4 arguments (3 given)' in 'lxml.etree._local_resolver' ignored As it says, this is ignored, but I get lxml.etree.XMLSyntaxError: None later on. This is with lxml 2.1.1 (on Windows XP) . I've also tried an older version (1.3.6) that was installed on my Ubuntu box, and that worked, however, after building 2.1.2 there, I get the same problem. I'm not sure whether this is an actual bug, or whether something has changed that is not clear from the docs. 2) resolve_string seems only to work for unicode strings and ASCII-only bytestrings. I would expect behaviour more consistent with etree.fromstring, so I'd assume this is a bug too? If you can confirm these are bugs, I'll file them in the tracker. greetings, Steven

xmlns / xmlns:xmlns inconsistency

Aaron Brady — 2008-09-12T00:08:22

Hello, I don't know if anyone checks this or cares. I tried to set a 'xmlns' attribute of a node. 'etree.tostring' produced an attribute name of 'xmlns:xmlns' instead. I found a workaround: call 'root.set( 'xmlns', 'urn:schemas-microsoft-com:office:spreadsheet' )' after creating the node. Full report, thanks for your time. from lxml import etree print "lxml.etree: ", etree.LXML_VERSION print "libxml used: ", etree.LIBXML_VERSION print "libxml compiled: ", etree.LIBXML_COMPILED_VERSION print "libxslt used: ", etree.LIBXSLT_VERSION print "libxslt compiled: ", etree.LIBXSLT_COMPILED_VERSION ''' lxml.etree: (2, 1, 0, 0) libxml used: (2, 6, 32) libxml compiled: (2, 6, 32) libxslt used: (1, 1, 23) libxslt compiled: (1, 1, 23) ''' ''' Description: Setting 'xmlns' attribute of tag produces 'xmlns:xmlns' instead. ''' ''' Possibly related issue: * With ``lxml.doctestcompare`` if you do ```` in your output, it will then be namespace-neutral (before the ellipsis was treated as a real namespace). ''' ''' Target: Current: ''' nns= { 'xmlns': 'urn:schemas-microsoft-com:office:spreadsheet', 'o': 'urn:schemas-microsoft-com:office:office', 'x': 'urn:schemas-microsoft-com:office:excel', 'ss': 'urn:schemas-microsoft-com:office:spreadsheet', 'html': 'http://www.w3.org/TR/REC-html40' } root= etree.Element( 'Workbook', nsmap= nns ) out= etree.tostring( root, pretty_print= True, xml_declaration=True ) print( out ) ''' Workaround: ''' nns= { 'o': 'urn:schemas-microsoft-com:office:office', 'x': 'urn:schemas-microsoft-com:office:excel', 'ss': 'urn:schemas-microsoft-com:office:spreadsheet', 'html': 'http://www.w3.org/TR/REC-html40' } root= etree.Element( 'Workbook', nsmap= nns ) root.set( 'xmlns', 'urn:schemas-microsoft-com:office:spreadsheet' ) out= etree.tostring( root, pretty_print= True, xml_declaration=True ) print( out ) ''' Output: ---------^------- '''

context parameter thread safety

Alex Klizhentas — 2008-09-08T15:23:10

_______________________________________________ lxml-dev mailing list lxml-dev< at >codespeak.net http://codespeak.net/mailman/listinfo/lxml-dev

lxml.html adds a default doctype to HTML documents

James Graham — 2008-09-07T20:15:59

In [2]: from lxml import html In [3]: t = html.fromstring("

Hello World") In [4]: docinfo = t.getroottree().docinfo In [5]: docinfo.public_id Out[5]: '-//W3C//DTD HTML 4.0 Transitional//EN' Is it possible to prevent this from occurring? I couldn't see anything in the API documentation but I might have been missing something obvious. Silently gaining incorrect data is annoying :)

Preventing XPath injection

Alex Klizhentas — 2008-09-06T12:18:42

_______________________________________________ lxml-dev mailing list lxml-dev< at >codespeak.net http://codespeak.net/mailman/listinfo/lxml-dev

lxml 2.1.2 and 2.0.9 released

Stefan Behnel — 2008-09-05T12:30:45

Hi, lxml 2.1.2 and 2.0.9 are on PyPI. Both are bug-fix releases for the stable and mature release series. They mainly fix a thread-related memory problem that was introduced in the last releases of both branches. Updating is recommended. The complete changelog follows below. Have fun, Stefan 2.0.9 (2008-09-05) Bugs fixed * Memory problem when passing documents between threads. * Target parser did not honour the recover option and raised an exception instead of calling .close() on the target. 2.1.2 (2008-09-05) Features added * lxml.etree now tries to find the absolute path name of files when parsing from a file-like object. This helps custom resolvers when resolving relative URLs, as lixbml2 can prepend them with the path of the source document. Bugs fixed * Memory problem when passing documents between threads. * Target parser did not honour the recover option and raised an exception instead of calling .close() on the target.

[Fwd: [Bug 263898] [NEW] Windows Installer crashes due to access violation]

Stefan Behnel — 2008-09-02T09:35:19

Hi, has anyone seem this before or could someone please test if this is reproducible on other machines? Thanks, Stefan -------------------------------------------------------------------------- Subject: [Bug 263898] [NEW] Windows Installer crashes due to access violation -------------------------------------------------------------------------- Public bug reported: I tried to install lxml-2.1.1.win32-py2.5.exe on my WinXP PC and got the attached crash from the installer. It happened after confirming all settings and clicking the "Next" button on the "Ready to install" page of the installer. Using Visual Studio Debugger to investigate crash : Callstack : ntdll.dll!_RtlEnterCriticalSection< at >4() + 0xb msvcr71.dll!_lock_file(void * pf=0x00000000) Line 236C format=0x0012d278, ...) Line 63 + 0x6C lxml-2.1.1.win32-py2.5.exe!00402ca8() user32.dll!77d48734() user32.dll!77d48bd9() user32.dll!77d541dc() user32.dll!77d541a9() user32.dll!77d53fd9() ntdll.dll!_RtlpFreeToHeapLookaside< at >8() + 0x26 ntdll.dll!_RtlFreeHeap< at >12() + 0x114 ntdll.dll!_RtlpFreeAtom< at >4() + 0x1b c5ffffff() Looking at the fprintf function I can see that "str" variable is NULL. _lock_file function using NULL pointer the access struture variable which eventually results in the reported crash. ** Affects: lxml Importance: Undecided Status: New

Use cssselect.py in Pyxer

Dirk Holtwick — 2008-08-31T16:24:56

Hi, I wrote (yet another) templating language for Python based on Genshi, since Genshi itself does not yet work on Google App Engine (GAE). Since Genshi supports XPath I was thinking about using your cssselect.py module together with it. First tests showed that this seems to work fine. Now I would like to ship a little bit modified version of cssselect.py with this new templating language called "Pyxer" http://code.google.com/p/pyxer/ so the users do not have to install the whole lxml package (which does not work with GAE anyways I suppose). Since Python "lxml" is under the BSD license and Pyxer under MIT license I think this should not be such a big problem as long as I add your copyright notices to the file. Am I right? Thanks for your excellent work Dirk

very long files with many XML entity refs

Moshe Cohen — 2008-08-28T23:45:56

_______________________________________________ lxml-dev mailing list lxml-dev< at >codespeak.net http://codespeak.net/mailman/listinfo/lxml-dev

questions about lxml xsd:include and xsd:import behavior

Santoro, Peter — 2008-08-28T12:00:55

_______________________________________________ lxml-dev mailing list lxml-dev< at >codespeak.net http://codespeak.net/mailman/listinfo/lxml-dev

Encoding again

Max Ivanov — 2008-08-24T22:21:29

Hi again! I'm unable to solve encoding problems myself, so I ask again here, hope someone have solution. Is there any way to force lxml to make element.text and element.tail to be exactly the same as in original text, without any encoding manipulation? Or to restore them to original state, i.e. maybe somewhere inside lxml there is a var which contain original encoding, so I could do elelemt.text.encode('...').?

Text obscured by subelement

Richard Baron Penman — 2008-08-24T11:19:18

_______________________________________________ lxml-dev mailing list lxml-dev< at >codespeak.net http://codespeak.net/mailman/listinfo/lxml-dev