gmane.comp.python.lxml.devel

lxml.objectify drops leading whitespace

Wolfgang Schnerring — 2013-05-24T12:03:57

Hello,

lxml.objectify drops leading whitespace before a tag:

'<p><a>foo</a></p>'

while lxml.etree handles the same situation just fine:

'<p> <a>foo</a></p>'

What's the reason for the different behaviour? Is this intentional? Most
importantly, how can I make lxml.objectify preserve whitespace correctly?

Thanks for your help,
Wolfgang

_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

lxml object architecture quandary

John W. Shipman — 2013-05-18T03:16:09

I have a design problem involving ElementTree.  I'd appreciate
any suggestions.  The problem is posted online here:

     http://johnwshipman.blogspot.com/

Best regards,
John Shipman (john< at >nmt.edu), Applications Specialist
New Mexico Tech Computer Center, Speare 146, Socorro, NM 87801
(575) 835-5735, http://www.nmt.edu/~john
   ``Let's go outside and commiserate with nature.''  --Dave Farber
_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

Question on 1) encoding/decoding and 2) importing xsd

Sangeeth Saravanaraj — 2013-05-15T23:12:52

I have the following snippet of code which validates the given xml data
against a given xsd.

<code>

from lxml import etree
from io import StringIO, BytesIO

with open("schema1.xsd", "r") as schema_file:
    schemaContent = schema_file.read()
with open("data.xml", "r") as xml_file:
    xmlContent = xml_file.read()

xmlschema_doc = etree.parse(StringIO(schemaContent.decode('iso8859_1')))
xml_doc = etree.parse(StringIO(xmlContent.decode('iso8859_1')))

xmlschema = etree.XMLSchema(xmlschema_doc)
print xmlschema.validate(xml_doc)

</code>

The above code usually works fine but not under the following scenarios.

*#1) ValueError: Unicode strings with encoding declaration are not
supported.*

I see ValueError when the .xsd or .xml file begins with an 'xml' element
which contains the 'encoding' attribute as follows

    <?xml version="1.0" encoding="UTF-8"?>

When I remove the 'encoding' attribute, the problem vanishes. For example:
<?xml version="1.0"?>

I referred the Python Standard Encodings -
http://docs.pyth

Parsing problem

Arnaud RIVET — 2013-05-15T12:42:01

Hi,

I am new with LXML and I have a problem after parsing my element : its 
architecture seems to have changed if I remove (or replace) the last 
child.
Here is my code and a screen of the console output.
I have looked up for solution but I still can't figure what I am doing 
wrong.
I would really appreciate someone's help !
(I am using LXML 3.2.1 with Python 2.6 on Windows)

#############################################################################################

from lxml import etree
from copy import deepcopy

def Write( file, element ):
        f = open( file, 'w' )
        f.write( etree.tostring( element, xml_declaration=True, 
encoding="ISO-8859-1", pretty_print = True ) )
        f.close()
        return 1

def ReadAndReturn( file ):
        lookup = etree.ElementDefaultClassLookup()
        parser = etree.XMLParser(recover = True)
        parser.set_element_class_lookup( lookup )
        mainTree = etree.parse( file, parser )
        return mainTree

# create a root element with 3 children
roo

lxml 3.2.1 released

Stefan Behnel — 2013-05-11T22:03:53

Hi everyone,

I just sent out lxml 3.2.1 as the first major bug-fix release for the
stable 3.2 release series.

You can get it from PyPI: http://pypi.python.org/pypi/lxml/3.2.1

The documentation is here: http://lxml.de/

Download:  http://lxml.de/files/lxml-3.2.1.tgz

Signature: http://lxml.de/files/lxml-3.2.1.tgz.asc

Changelog: http://lxml.de/3.2/changes-3.2.1.html

GitHub:
https://github.com/lxml/lxml/commit/90afe19b11c4fd1df8998b254d8a60b550aa56d0

This release was built using Cython 0.19.1. Note that the build no longer
uses Cython, even if it is installed. Recompilation of the sources has to
be requested explicitly with the setup.py option "--with-cython".

If you are interested in commercial support or customisations for the lxml
package, please contact me directly.

Have fun,

Stefan


3.2.1 (2013-05-11)
==================

Features added
--------------

* The methods ``apply_templates()`` and ``process_children()`` of XSLT
  extension elements have gained two new boolean options ``elements_only``

Comments causing malloc/bus error/segmentation fault

Ed Singleton — 2013-05-10T11:22:51

Apologies if this isn't the right place to send this, but I'm not sure what to do with it.

I put it on StackOverflow where it has been pretty much ignored (if anyone would prefer to answer it there).
http://stackoverflow.com/questions/16459539/malloc-and-bus-error-in-python-with-lxml-comments

I've been having lots of problems with comments crashing my interpreter in lxml.  A simplest case that causes malloc is:

```
import lxml
import lxml.builder
import lxml.html

print "Causing malloc"
builder = lxml.builder.ElementMaker()
el = builder.div()
el.append(lxml.html.HtmlComment("foo"))
```
I've tested this on python2.6 and python3.2 (with just a change to the print statements).  It causes segmentation faults on python3.2

A more full example that also demonstrates more of what I've been trying to do is:

```
import lxml
import lxml.builder
import lxml.html
import lxml.etree

print "Causing Bus Error"
class HtmlElement(lxml.html.HtmlElement):
    pass

class HtmlElementLookup(lxml.html.HtmlElementClassLookup):

Patch for losing namespace on attributes

Tyler Spivey — 2013-05-04T04:53:46

I tried patching this. I'm not sure this is the right way of doing things, but it passes
the existing tests as well as the one added for it.

diff -r 452a2705665b -r 41fee8a923a1 src/lxml/proxy.pxi
--- a/src/lxml/proxy.pxiSun Apr 28 20:42:33 2013 +0200
+++ b/src/lxml/proxy.pxiFri May 03 17:07:35 2013 -0700
< at >< at > -287,7 +287,7 < at >< at >
     while c_nsdef[0] is not NULL:
         c_ns = tree.xmlSearchNsByHref(
             c_element.doc, c_element.parent, c_nsdef[0].href)
-        if c_ns is NULL:
+        if c_ns is NULL or c_ns.prefix is NULL:
             # new namespace href => keep and cache the ns declaration
             _appendToNsCache(c_ns_cache, c_nsdef[0], c_nsdef[0])
             c_nsdef = &c_nsdef[0].next
< at >< at > -340,6 +340,7 < at >< at >
     cdef xmlNs* c_ns_next
     cdef xmlNs* c_nsdef
     cdef xmlNs* c_del_ns_list
+    cdef xmlNs* c_default_ns = NULL
     cdef size_t i, proxy_count = 0
 
     if not tree._isElementOrXInclude(c_element):
< at >< at > -366,12 +367,18 < at >< at >
 
         # 2) make sure the namespaces of an elemen

UnicodeDecodeError in lxml

Joaquin Cuenca Abela — 2013-05-03T13:30:41

Hi,

we are parsing lots of html files with lxml, and we're facing some problems
with their encoding.

Short story, we want to parse an HTML with an encoding even if the HTML is
not fully correctly encoded. For example, we may have a 500K file correctly
encoded in UTF-8 but with a single word encoded in iso-latin-1.

lxml gives a UnicodeDecodeError when we do:

self._html = lxml.html.fromstring(
    html_text,
    parser=lxml.html.HTMLParser(
                       encoding='utf-8',
                       remove_blank_text=True,
                       remove_comments=True,
                       remove_pis=True))

It seems the only way around it is to sanitize the text before sending it
to lxml, which is a bit slow:

self._html = lxml.html.fromstring(
    html_text.decode('utf-8', errors='ignore'),
    parser=lxml.html.HTMLParser(
                       encoding='unicode',
                       remove_blank_text=True,
                       remove_comments=True,
                       remove_pis=True))

Thi

bidirectional python <-> xml transformations?

Eric Levy — 2013-05-02T20:28:27

Hello,

I am experience an issue writing data python data that had been 
generated by an XPath query.  It appears that lxml uses some pythonic 
magic to transform between Python objects and XML data.  It also 
appears, unfortunately that the same data that was generated by the 
library cannot be written back to XML.

Specifically, I am seeking to develop a simple web page that is based on 
lxml applying XSLT templates.  Human readable text, as well as 
high-level formatting information, is kept in XML documents.  XSLT 
templates transform the documents to HTML pages.  Because 
internationalization is necessary, text is kept separate documents.  The 
particular document used depends on the locale, which is determined at 
run time by the Python application.

To satisfy these requirements, I have attempted to use the XSLT 
extensions feature of lxml.  Before processing begins, I parse the XML 
document containing human-readable text.  I then perform the 
transformation on the main XML document, passing an exten

How to control the processing of newlines in etree.html xpath text() function?

Olivier de Mirleau — 2013-04-30T07:56:33

Hi List,

I recently upgraded from linux Fedora 17 to 18, and am facing a change
in functionality of the lxml xpath text() function. Here's an example:

from io import BytesIO
from lxml import etree

myHtmlString = \
    '<!doctype html public "-//w3c//dtd html 4.0 transitional//en">\r\n'+\
    '<html>\r\n'+\
    '<head>\r\n'+\
    '   <title> a b c </title>\r\n'+\
    '</head>\r\n'+\
    '<body/>\r\n'+\
    '</html>\r\n'
myFile = BytesIO(myHtmlString)
myTree = etree.parse(myFile, etree.HTMLParser())
myTextElements = myTree.xpath("//text()")
myFullText = ''.join([myEl for myEl in myTextElements])

print repr(myFullText)


Under F17 that piece of code will write
' a b c '
whereas F18 produces
'\r\n\r\n    a b c \r\n\r\n\r\n'


The version specifications are as follows:
f17:
Python              : sys.version_info(major=2, minor=7, micro=3,
releaselevel='final', serial=0)
lxml.etree          : (2, 3, 5, 0)
libxml used         : (2, 7, 8)
libxml compiled     : (2, 7, 8)
libxslt used        : (1, 1, 26)
libxslt c

lxml 3.2.0 released

Stefan Behnel — 2013-04-28T19:08:24

Hi everyone,

I just released lxml 3.2.0 as a major bug-fix release. Much of the work
(fixing and analysing bugs) that went into this release was done by the
participants of the lxml bugathon this weekend. Thanks everyone!

You can get it from PyPI: http://pypi.python.org/pypi/lxml/3.2.0

The documentation is here: http://lxml.de/

Download:  http://lxml.de/files/lxml-3.2.0.tgz

Signature: http://lxml.de/files/lxml-3.2.0.tgz.asc

Changelog: http://lxml.de/3.1/changes-3.2.0.html

GitHub:
https://github.com/lxml/lxml/commit/8fc99f8ef3e2f12e74fb407ae62c5db50cfe7b92

This release was built using Cython 0.19. Note that the build no longer
uses Cython, even if it is installed. Recompilation of the sources has to
be requested explicitly with the setup.py option "--with-cython".

If you are interested in commercial support or customisations for the lxml
package, please contact me directly.

Have fun,

Stefan


3.2.0 (2013-04-28)
==================

Features added
--------------

Bugs fixed
----------

* LP#690319: L

Creating an lxml portable distribution

Szymon Pyżalski — 2013-04-20T11:49:17

Hello!

I need to install a current release of lxml into a virtualenv on a VPS.
This VPS however has some memory quota that doesn't allow me to build it
package there. I tried therefore to build a distribution on my laptop
(the same architecture, but other linux distro). I used:

$ CFLAGS='-fPIC' python setup.py bdist_egg --static_deps

After installing the egg (with easy_install) on the server I get this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File
"/home/pythonista/www/virtualenv/pc-qa/lib/python3.2/site-packages/lxml-3.1.2-py3.2-linux-x86_64.egg/lxml/etree.py",
line 7, in <module>
    __bootstrap__()
  File
"/home/pythonista/www/virtualenv/pc-qa/lib/python3.2/site-packages/lxml-3.1.2-py3.2-linux-x86_64.egg/lxml/etree.py",
line 6, in __bootstrap__
    imp.load_dynamic(__name__,__file__)
ImportError:
/home/pythonista/www/virtualenv/pc-qa/lib/python3.2/site-packages/lxml-3.1.2-py3.2-linux-x86_64.egg/lxml/etree.cpython-32mu.so:
cannot open shared object file: No such file

lxml 3.1.2 released

Stefan Behnel — 2013-04-12T06:06:14

Hi everyone,

I just released lxml 3.1.2. This is a pure bug-fix release for the stable
3.1 series.

You can get it from PyPI: http://pypi.python.org/pypi/lxml/3.1.2

The documentation is here: http://lxml.de/

Download:  http://lxml.de/files/lxml-3.1.2.tgz

Signature: http://lxml.de/files/lxml-3.1.2.tgz.asc

Changelog: http://lxml.de/3.1/changes-3.1.2.html

GitHub:
https://github.com/lxml/lxml/commit/76c9042511bfee787efad2d2f5cbdb850c01fbd6

This release was built using Cython 0.18. Note that the build no longer
uses Cython, even if it is installed. Recompilation of the sources has to
be requested explicitly with the setup.py option "--with-cython".

If you are interested in commercial support or customisations for the lxml
package, please contact me directly.

Have fun,

Stefan


3.1.2 (2013-04-12)
==================

Bugs fixed
----------

* LP#1136509: Passing attributes through the namespace-unaware API of
  the sax bridge (i.e. the ``handler.startElement()`` method) failed
  with a ``TypeError``.  Patc

Bugathon on April 27th in München

Stefan Behnel — 2013-04-12T05:49:29

Hi all,

the official date for the bugathon is

Saturday, April 27th, starting at 10 AM

It's taking place in München, Oettingenstraße 67. That's the usual meeting
place of the local Python User Group.

There were 8 signups so far, which I think is a reasonable size for this
event. Since there was only one request for participation through IRC, I
think it's best to drop the IRC participation completely as it would just
add overhead to the otherwise local communication.

NOTE: For those who signed up but haven't sent me an e-mail yet: please
send me a private (i.e. direct, off-list) reply to this e-mail so that I
have your e-mail address for the further planning.

Stefan
_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

performance decrease over 10000 rows with lpod(uses lxml) (open office library for python)

yiğit bul — 2013-04-05T15:10:19

Hey everyone ,

i am using lpod pytohn library to manipulate (just read for now actually)
.ods files.

lpod is using lxml behind. We have a problem with reading files that have
so many rows , but it looks like an exponential bottleneck. I tracked the
code down in the lpod library and reached where theres a xpath query
running to get all the rows.

xpath("table:table-row") operation (thats the exact operation):

10.000 rows , getting the whole rows is 0.1 seconds
12.000 rows , getting the whole rows is 0.5 seconds
15.000 rows , getting the whole rows is 1.3 seconds
20.000 rows , getting the whole rows is 7 seconds
30.000 rows , getting the whole rows is 20 seconds
60.000 rows , getting the whole rows is "i dont know how long" .

is this usual ?





--
_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

anyone in for an lxml bugathon?

Stefan Behnel — 2013-04-01T08:21:55

Hi everyone,

since the list of tickets in lxml's bug tracker just keeps ever growing,
I'd like to organise a little bugathon to get rid of some of them and to at
least look through the unprocessed ones and figure out if they are valid
issues and what to do about them.

https://bugs.launchpad.net/lxml

There's lots of different tickets about various parts of lxml, reaching
from opinions about unexpected behaviour over inappropriate APIs or missing
features all the way to crashes and OS specific problems. Sometimes, it may
just be the documentation that needs a little tweaking, sometimes it may
turn out that only a week worth of programming would get something
resolved. In a few cases, it's just unclear what the person reporting the
issue was trying to say and if there really is a problem. Or it may be
triggered by a bug in libxml2 or libxslt, in which case someone would need
to forward the bug to the respective bug tracker and/or write an e-mail to
their mailing list about it.

So I'm sure there's something

Namespaced attributes losing their namespace

Tyler Spivey — 2013-03-31T01:14:51

In the below test script, I'm not getting the result I expect.
Is this a bug in lxml? If not, what am I doing wrong?
I'm using lxml 3.1.1 on win32, downloaded from PyPi.

What I get:

<root xmlns="url1">
   <test attrib="value"/>
</root>

I expect attrib to not lose its namespace, as shown when I re-parse the 
output:

 >>> lxml.etree.parse('test.xml').getroot()[0].attrib
{'attrib': 'value'}

If I swap the append and the attribute setting, things work:
   <test xmlns:ns0="url1" ns0:attrib="value"/>

 >>> lxml.etree.parse('test.xml').getroot()[0].attrib
{'{url1}attrib': 'value'}

If this is intended behaviour, I'll have to see how to fit that into my 
design, since my functions return elements ready for appending.


from lxml import etree

nsmap = {None: 'url1'}
root = etree.Element('{url1}root', nsmap=nsmap)
test = etree.Element('{url1}test')
test.attrib['{url1}attrib'] = 'value'
root.append(test)
print etree.tostring(root, encoding='utf-8', pretty_print=True)
________________________________________________

How to pass xmlSaveOption flags to etree.tostring?

Marcello Perathoner — 2013-03-30T16:10:26

I need to serialize XHTML 1.1 without minimized empty elements.

Specifically, I want to produce EPUB 2 files, which require XHTML 1.1, 
but some user agents, like epubreader for firefox, load the XHTML files 
from disk, whereas firefox uses the HTML parser and makes a big mess of 
minimized elements.

So I wonder if there is a way to pass the option flags in libxml 
xmlSaveOption to etree.tostring ().

In case there is no such way, I humbly request the feature to be added.




Regards

lxml 3.1.1 released

Stefan Behnel — 2013-03-29T21:21:13

Hi everyone,

I just released lxml 3.1.1. This is a pure bug-fix release for the stable
3.1 series.

You can get it from PyPI: http://pypi.python.org/pypi/lxml/3.1.1

The documentation is here: http://lxml.de/

Download:  http://lxml.de/files/lxml-3.1.1.tgz

Signature: http://lxml.de/files/lxml-3.1.1.tgz.asc

GitHub:
https://github.com/lxml/lxml/commit/98c8562f7d659b49997c136947afdf6bf835d5ab

This release was built using Cython 0.18. Note that the build no longer
uses Cython, even if it is installed. Recompilation of the sources has to
be requested explicitly with the setup.py option "--with-cython".

If you are interested in commercial support or customisations for the lxml
package, please contact me directly.

Have fun,

Stefan


3.1.1 (2013-03-29)
==================

Features added
--------------

Bugs fixed
----------

* LP#1160386: Write access to ``lxml.html.FormElement.fields`` raised
  an AttributeError in Py3.

* Illegal memory access during cleanup in incremental xmlfile writer.

Other changes
---

Solution for Inkscape osx 10.7 and lxml glitch

Kurt Olsen — 2013-03-10T22:04:35

Well folks, if anybody else comes here complaining about this issue for the
osx 10.7.5 Inkscape 48 problem - this worked. Again, thanks for your time...

I have written a workaround patch and made it available for download (as
part of the Eggbot extensions for Inkscape), for anyone needs a short-term
workaround for this issue-- that is, before an official solution is
released by the Inkscape developers.

You can download it here:
http://code.google.com/p/eggbotcode/downloads/detail?name=EggBot2.2.2.r2.mpkg.zip

Full disclosure:
A. This is a totally unofficial hack, not endorsed in any way by the
Inkscape developers. :)

B. The download is a standard Mac installer package for the Eggbot
extensions for Inkscape, which installs the following software:

1) The Eggbot extensions for Inkscape, a set of Inkscape Extensions for
creating drawings for and operating an Eggbot,
2) The Hershey Text extension for Inkscape, a "render" extension for
drawing stroke-based text, and
3) The patch to Inkscape.app, for version 0.

Installation on Mac insanely complex

Kurt Olsen — 2013-03-10T13:15:32

Aloha,

I'm trying to do 3d printing. Do to that I need to use Inkscape, and one
it's plugins, which demands lxml. And I cannot get it installed. I'm an
end-user. I know nothing of 'ports' or 'macports' of building it or this or
that or the million other complex things you folks and google are talking
about. All I know is this thing 'lxml' that you folks maintain is stopping
me cold because I can't install it on

OSX 10.7.5
Python 2.7.1
Inkscape 0.48

Any help would be greatly appreciated because right now LXML is standing
between me and $2500 worth of 3d printer!

Thanks in advance,
Kurt
_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml