gmane.comp.python.lxml.devel

Re: Error seen when running unit tests

Stefan Behnel — 2013-05-25T08:28:07

Am 22.05.2013 19:54, schrieb Lindykid:

My educated guess is that it's an issue with libxml2. The 2.7.x versions
below 2.7.8 are known to have varying minor issues. If you can't
(easily) upgrade your system library, consider a static lxml build,
preferably using the latest libxml2 2.9.x release.

Alternatively, test your own code with what you have and if that works,
it's unlikely to magically start failing without modifications. Bugs in
libxml2 often just impact specific features (such as push parsing
above), not all of lxml.

Stefan

_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

Changing encoding while parsing an HTML string

Michael O'Leary — 2013-05-24T19:22:56

If I understand how the XML parser works, if it parses an XML string that
begins with a header line like

<?xml version="1.0" encoding="utf-8"?>

it will parse the rest of the XML string using the encoding of the encoding
it has found. So first of all, does the XML parser do this?

If so, I'd like to know if something like this can be done, either
automatically or manually, with the HTML parser.

I have been parsing HTML strings that have been obtained from a random
sampling of web sites, and I am wondering if there is a way to do something
similar with these HTML strings. Typically, these HTML strings do not start
with a header like, but most of them do have a meta tag of the form

<meta charset="utf-8">

or

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" >

I wouldn't expect that the HTML parser reads these tags and automatically
sets the encoding to be used while parsing the rest of the HTML string (but
I might as well ask: does it?). I could include code in my target's start
tag handl

Re: lxml.objectify drops leading whitespace

Holger Joukl — 2013-05-24T16:33:31

Hi,


It is intentional. For data binding the text content of elements with
children is (usually) irrelevant.

IIRC this only drops whitespace for elements with children, not for
elements
with text-only content.

You should be able to change objectify's behaviour by changing it's
default parser using makeparser() and set_default_parser:

def set_default_parser(new_parser = None):
    u"""set_default_parser(new_parser = None)

    Replace the default parser used by objectify's Element() and
    fromstring() functions.

    The new parser must be an etree.XMLParser.

    Call without arguments to reset to the original parser.
    """

def makeparser(**kw):
    u"""makeparser(remove_blank_text=True, **kw)

    Create a new XML parser for objectify trees.

    You can pass all keyword arguments that are supported by
    ``etree.XMLParser()``.  Note that this parser defaults to removing
    blank text.  You can disable this by passing the
    ``remove_blank_text`` boolean keyword option yourself.
    """

lxml.objectify drops leading whitespace

Wolfgang Schnerring — 2013-05-24T12:03:57

Hello,

lxml.objectify drops leading whitespace before a tag:

'<p><a>foo</a></p>'

while lxml.etree handles the same situation just fine:

'<p> <a>foo</a></p>'

What's the reason for the different behaviour? Is this intentional? Most
importantly, how can I make lxml.objectify preserve whitespace correctly?

Thanks for your help,
Wolfgang

_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

Error seen when running unit tests

Lindykid — 2013-05-22T17:54:11

Deal lxml Developers,

  I want to use your library for some projects that I will need XML parsing and XSD verification.  I am attempting to install lxml 3.2.1 onto a RHEL 6.1 server which has python 2.6.6 installed.  I did not use pip to build and install lxml, rather I used the 'python setup.py install'.  Everything seemed to build and install fine.  I was able to run through a few tutorials with out any issue.   
  My problem came when I decided to try and run the unit tests that you have provided.  I get one failure (see bottom of this E-Mail) when I run the 'make test' or 'python test.py -p -v' command.  I have tried this on lxml versions of 3.1.2, 3.2.0, and 3.2.1 and I see the same test error on each release.  I have attempted to sower the web, and the lxml mailing list archives to see if I could find any information on what is causing this test to fail, but I have come up empty.  I am hoping that one of you can tell me what I am doing wrong, or what I can to to get this test to pass.

Re: lxml object architecture quandary

Holger Joukl — 2013-05-21T08:19:40

Hi,


Have you looked into using custom element classes (
http://lxml.de/element_classes.html)
so your wrapper classes would turn into real Elements?

Provided that you
- don't need stdlib ElementTree compatibility (lxml-only feature)
- don't need internal state in your custom element instances or store
internal state in the XML tree
- can define a class lookup scheme to have lxml instantiate your custom
element classes
(by tag name, attributes, namespace, ...) when parsing

Could also be combined with using xml.objectify
(http://lxml.de/objectify.html) for a more
data binding-oriented element API, but that's a matter of preference.

Best regards,
Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart

_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

Re: lxml object architecture quandary

Burak Arslan — 2013-05-18T09:38:17



Hi,

About this bit above: 
https://github.com/arskom/spyne/blob/master/examples/xml_utils.py

Best,
Burak

_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

lxml object architecture quandary

John W. Shipman — 2013-05-18T03:16:09

I have a design problem involving ElementTree.  I'd appreciate
any suggestions.  The problem is posted online here:

     http://johnwshipman.blogspot.com/

Best regards,
John Shipman (john< at >nmt.edu), Applications Specialist
New Mexico Tech Computer Center, Speare 146, Socorro, NM 87801
(575) 835-5735, http://www.nmt.edu/~john
   ``Let's go outside and commiserate with nature.''  --Dave Farber
_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

Re: Question on 1) encoding/decoding and 2) importing xsd

Holger Joukl — 2013-05-16T09:25:57

Hi,

supported.


Don't read the file, decode and parse from the unicode string; leave
parsing to lxml in one step:

xml_doc = etree.parse(<file object or file name/path/uri>)


You'll also avoid this problem when using etree.parse() directly:

$ python2.7 -i -c'from lxml import etree, objectify; root = etree.parse
("XMLSchemaProps/main.xsd"); schema = etree.XMLSchema(root)'
True
$ python2.7 -i -c'from lxml import etree, objectify; root =
etree.fromstring(open("XMLSchemaProps/main.xsd").read()); schema =
etree.XMLSchema(root)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "xmlschema.pxi", line 105, in lxml.etree.XMLSchema.__init__
(src/lxml/lxml.etree.c:132771)
lxml.etree.XMLSchemaParseError: failed to load external entity
"imported.xsd"

Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart

_________________________________________________________________
Mailing list for th

Re: Question on 1) encoding/decoding and 2) importing xsd

Ed Singleton — 2013-05-16T09:03:48

The `xmlContent.decode('iso8859_1')` turns the string into unicode, so you are passing unicode into the parser


Here you are saying that the content is a string encoded as utf-8, but it isn't.  It's unicode.  That's why you get the error.


Because you are no longer claiming that the content is encoded.


That you aren't getting encoding errors, so ignoring them won't help.  It's just that the parser is being strict and won't let you lie to it.

Either don't decode, and then pass in a string, or don't have an encoding declaration in the doc

Ed_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

Question on 1) encoding/decoding and 2) importing xsd

Sangeeth Saravanaraj — 2013-05-15T23:12:52

I have the following snippet of code which validates the given xml data
against a given xsd.

<code>

from lxml import etree
from io import StringIO, BytesIO

with open("schema1.xsd", "r") as schema_file:
    schemaContent = schema_file.read()
with open("data.xml", "r") as xml_file:
    xmlContent = xml_file.read()

xmlschema_doc = etree.parse(StringIO(schemaContent.decode('iso8859_1')))
xml_doc = etree.parse(StringIO(xmlContent.decode('iso8859_1')))

xmlschema = etree.XMLSchema(xmlschema_doc)
print xmlschema.validate(xml_doc)

</code>

The above code usually works fine but not under the following scenarios.

*#1) ValueError: Unicode strings with encoding declaration are not
supported.*

I see ValueError when the .xsd or .xml file begins with an 'xml' element
which contains the 'encoding' attribute as follows

    <?xml version="1.0" encoding="UTF-8"?>

When I remove the 'encoding' attribute, the problem vanishes. For example:
<?xml version="1.0"?>

I referred the Python Standard Encodings -
http://docs.pyth

Re: Parsing problem

Holger Joukl — 2013-05-15T16:05:10

Hi,

#############################################################################################


'None'
pretty_print=True))[1].tail
'
  '

See http://lxml.de/tutorial.html#elements-contain-text for text & tail
attributes in the lxml docs.

Not related to lxml (but for once I couldn't resist ;-)):

encoding="ISO-8859-1", pretty_print = True ) )

- consider following Python style guide (PEP 8) recommendations wrt
function/method names, whitespace
and parentheses (unless your codebase or organization dictates otherwise).
E.g. rather use write() instead of Write(),...

- consider using context management for file access to make sure your file
gets properly closed,
whatever might fail in the code after opening the file:
...     f.write('foobar')
...
<closed file 'foo.bar', mode 'w' at 0x39f7b0>

Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart

_______________________________________________________________

Re: Parsing problem

jens quade — 2013-05-15T15:13:56

On 15.05.2013, at 14:42, Arnaud RIVET <arnaud.rivet< at >cdr.hutchinson.fr> wrote:

Hi,


What output do you expect? 

Except from some more whitespace, inserted during tostring because of the pretty_print=true setting, the output seems to be the same. 

By writing with pretty_print and then parsing the result, the whitespace becomes part of the XML document, in this case child2.tail is probably '\n  '. And this indents the </root> in your third case, because child3 is gone.



'<a>\n  <b/>\n  <c/>\n</a>\n'
'\n  '

If you do not want this behaviour, do not use pretty_print.



jens




_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

Parsing problem

Arnaud RIVET — 2013-05-15T12:42:01

Hi,

I am new with LXML and I have a problem after parsing my element : its 
architecture seems to have changed if I remove (or replace) the last 
child.
Here is my code and a screen of the console output.
I have looked up for solution but I still can't figure what I am doing 
wrong.
I would really appreciate someone's help !
(I am using LXML 3.2.1 with Python 2.6 on Windows)

#############################################################################################

from lxml import etree
from copy import deepcopy

def Write( file, element ):
        f = open( file, 'w' )
        f.write( etree.tostring( element, xml_declaration=True, 
encoding="ISO-8859-1", pretty_print = True ) )
        f.close()
        return 1

def ReadAndReturn( file ):
        lookup = etree.ElementDefaultClassLookup()
        parser = etree.XMLParser(recover = True)
        parser.set_element_class_lookup( lookup )
        mainTree = etree.parse( file, parser )
        return mainTree

# create a root element with 3 children
roo

lxml 3.2.1 released

Stefan Behnel — 2013-05-11T22:03:53

Hi everyone,

I just sent out lxml 3.2.1 as the first major bug-fix release for the
stable 3.2 release series.

You can get it from PyPI: http://pypi.python.org/pypi/lxml/3.2.1

The documentation is here: http://lxml.de/

Download:  http://lxml.de/files/lxml-3.2.1.tgz

Signature: http://lxml.de/files/lxml-3.2.1.tgz.asc

Changelog: http://lxml.de/3.2/changes-3.2.1.html

GitHub:
https://github.com/lxml/lxml/commit/90afe19b11c4fd1df8998b254d8a60b550aa56d0

This release was built using Cython 0.19.1. Note that the build no longer
uses Cython, even if it is installed. Recompilation of the sources has to
be requested explicitly with the setup.py option "--with-cython".

If you are interested in commercial support or customisations for the lxml
package, please contact me directly.

Have fun,

Stefan


3.2.1 (2013-05-11)
==================

Features added
--------------

* The methods ``apply_templates()`` and ``process_children()`` of XSLT
  extension elements have gained two new boolean options ``elements_only``

Re: Patch for losing namespace on attributes

Stefan Behnel — 2013-05-11T21:51:12

Stefan Behnel, 05.05.2013 08:59:

https://github.com/lxml/lxml/commit/7059c6958d90399006905973af5eb2874b1982c9

Stefan

_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

Re: Comments causing malloc/bus error/segmentation fault

Stefan Behnel — 2013-05-10T21:40:25

Ed Singleton, 10.05.2013 18:09:

Fixed as well. Thanks for insisting. :)

https://github.com/lxml/lxml/commit/8867ff26975d2a3e2822b15a7f41c404c3521017

Stefan

_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

Re: Comments causing malloc/bus error/segmentation fault

Ed Singleton — 2013-05-10T16:09:51

Thank you for fixing it so quickly.  I've checked out the master branch, and can confirm that it no longer breaks.

The other test case is still causing a bus error though.  I assume it's an unrelated issue.

I'm trying to create a html builder that returns a subclass of HtmlElement (which in my normal code has lots of additional methods). 
https://github.com/Singletoned/wiseguy/blob/master/wiseguy/html.py

It seems that returning a subclass of HtmlElement in the HtmlElementLookup causes a BusError, whereas returning the HtmlElement doesn't.  I'm quite confused as it's printing the HtmlComment that causes the error.

An example is below.

```
import lxml, lxml.builder, lxml.html, lxml.etree

print "Causing Bus Error"

class HtmlElement(lxml.html.HtmlElement):
    pass

class HtmlElementLookup(lxml.html.HtmlElementClassLookup):
    def lookup(self, node_type, document, namespace, name):
        if node_type == 'comment':
            return lxml.html.HtmlComment
        else:
            return lxml.html.Html

Re: Comments causing malloc/bus error/segmentation fault

Stefan Behnel — 2013-05-10T13:08:15

Ed Singleton, 10.05.2013 13:22:

Perfect place to post it.



Thanks for this excellent test case. That truly is one of those "I can't
believe no-one found this yet" kind of bugs. It's been there for over three
years now. (Or maybe people found it and just didn't report it, so thanks
for the report.)

https://github.com/lxml/lxml/commit/46e3b2570b6e63cfcb2c8bfe11467466b93b81d1

I'll see that I can get out a bug-fix release soon.

Stefan

_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.webfaction.com/listinfo/lxml

Comments causing malloc/bus error/segmentation fault

Ed Singleton — 2013-05-10T11:22:51

Apologies if this isn't the right place to send this, but I'm not sure what to do with it.

I put it on StackOverflow where it has been pretty much ignored (if anyone would prefer to answer it there).
http://stackoverflow.com/questions/16459539/malloc-and-bus-error-in-python-with-lxml-comments

I've been having lots of problems with comments crashing my interpreter in lxml.  A simplest case that causes malloc is:

```
import lxml
import lxml.builder
import lxml.html

print "Causing malloc"
builder = lxml.builder.ElementMaker()
el = builder.div()
el.append(lxml.html.HtmlComment("foo"))
```
I've tested this on python2.6 and python3.2 (with just a change to the print statements).  It causes segmentation faults on python3.2

A more full example that also demonstrates more of what I've been trying to do is:

```
import lxml
import lxml.builder
import lxml.html
import lxml.etree

print "Causing Bus Error"
class HtmlElement(lxml.html.HtmlElement):
    pass

class HtmlElementLookup(lxml.html.HtmlElementClassLookup):

Re: bidirectional python <-> xml transformations?

Stefan Behnel — 2013-05-08T06:10:45

Hi,

please keep this on-list.

Eric Levy, 08.05.2013 07:52:

Please read again what I wrote.



It's how XSLT works, regardless of the source of the data. If you use the
document() function in your stylesheet, it will do exactly the same thing.
Everything else would be horribly inconsistent. And apply_templates() works
*exactly* like <xsl:apply-templates />, for the same reason.



Again, the source doesn't matter.



Ah, so that's the source of the confusion. You are misreading
apply_templates() as <xsl:call-template />. No, that's not what it is.

I think what you really want is to explicitly add some content to the
output_parent. You have both in your hands, so use the ET API to do that in
exactly the way you want it to be. Or add a straight-forward recursive
<xsl:copy> to your stylesheet to do it for you and call apply_templates().

Stefan

_________________________________________________________________
Mailing list for the lxml Python XML toolkit - http://lxml.de/
lxml< at >lxml.de
https://mailman-mail5.w