gmane.lisp.lib.cxml.devel

Patch for ASDF 3

Faré — 2013-01-29T18:01:55

The way you look for the path of catalog.dtd doesn't work in ASDF 3.
Here is a supported way that works with all versions of ASDF 2 and later.

I'll try to improve backward compatibility, but you may nonetheless
want to apply the patch.
Also, I believe that three years afterwards, and
with each and every implementation providing ASDF 2,
it is safe to remove the #-asdf2 code around your codebase.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Work harder! Millions on welfare depend on you.

Possible bug in rune-dom::adjust-vector-exponentially

Camille Troillard — 2012-12-05T15:46:29

Hi,

I have a process that is stuck in the following function:

;; from dom-impl.lisp
(defun adjust-vector-exponentially (vector new-dimension set-fill-pointer-p)
  (let ((d (array-dimension vector 0)))
    (when (< d new-dimension)
      (loop
  do (setf d (* 2 d))
  while (< d new-dimension))
      (adjust-array vector d))
    (when set-fill-pointer-p
      (setf (fill-pointer vector) new-dimension))))


Apparently, adjust-vector-exponentially assumes d > 0, but this function has been called from (METHOD SAX:CHARACTERS (RUNE-DOM::DOM-BUILDER T)) with an empty string (d = 0).  The loop obviously shows it will never exit the function.

Adding the following before the loop statement fixes the problem:

      (when (= d 0)
        (incf d))


Best Regards,
Camille

Incorrect character with code 12 in output?

Camille Troillard — 2012-12-05T16:11:16

Hello,

I have a database with text data which sometimes is not proper to valid XML.
For example, there is a field sometimes containing the character new-page or ^L in emacs (ASCII code = 12).

While CXML does not complain and outputs cmxl:text as it was given, another XML parser complains that this is not valid XML.

PCDATA invalid Char value 12

I am expecting CXML to produce valid output, is there something I missed to make it more strict?

I am using LispWorks 64 bits.


Thanks!

Best Regards,
Cam

Klacks bug with &# entities on CCL

Andreas Fuchs — 2012-09-10T00:33:08

Hi there,

The following code breaks in CCL (Version 1.8-r15286M  (DarwinX8632))
with the most recent cxml from quicklisp (cxml-20110619-git):

(klacks:with-open-source (source (cxml:make-source "<value>&#32;987lkjasd</value>"))
  (klacks:consume source)
  (klacks:consume source)
  (klacks:consume source))

Here's the SLIME backtrace I get from evaluating that form:

  0: ((SETF RUNES:%RUNE) #\  "" 0)
  1: (CXML::READ-TOKEN-3 #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK ...))
  2: (CXML::PEEK-TOKEN #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK ...))
  3: (CXML::KLACKS/CONTENT #<CXML::CXML-SOURCE #xD7DE306> #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK ...) #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL ..))
  4: (CXML::KLACKS/ELEMENT-2 #<CXML::CXML-SOURCE #xD7DE306> #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK ...) (("") ("xmlns" . "http://www.w3.org/2000/xmlns/") ..) #<CCL:COMPILED-L..
  5: (CXML::FILL-SOURCE #<CXML

Bug in DTD parsing?

Bart Botta — 2012-08-28T22:58:01

(flex:with-input-from-sequence (s (babel:string-to-octets
                                   "<![IGNORE[ ]]><!-- -->"
                                   :encoding :utf-8))
  (cxml:parse-dtd-stream s))

errors with
Document not well-formed: Unexpected character #\!.
Context:
  Line 1, column 17 in NIL

   [Condition of type CXML:WELL-FORMEDNESS-VIOLATION]

but parses OK with whitespace between ]]> and <!--

(for a real example, see line 396 in http://docbook.org/xml/4.3/dbpoolx.mod)

it looks like P/IGNORE-SECT in cxml/xml/xml-parse.lisp should unread 1
rune after matching ]]> (or exit the loop sooner so it doesn't read an
extra).

Re: validating XML: document

Alexandre Rademaker — 2012-07-12T12:22:17

Hello David,

Thank you for your message. Indeed, your tip worked and correctly
omitted the warnings. As I said before, with some help and good
description about the possible root cause of the bug, I would be glad
to help you fix the bug in CXML code.

Best,

Alexandre Rademaker
http://arademaker.github.com/

On Wed, Jul 11, 2012 at 11:33 AM, David Lichteblau <david< at >lichteblau.com> wrote:

Re: validating XML: document

David Lichteblau — 2012-07-11T14:33:29

Hi,

Quoting Alexandre Rademaker (arademaker< at >gmail.com):

sorry for the late response.  The warnings are a cxml bug I haven't
fixed yet (sorry about that, too :-)).  Until then, my recommendation is
to use the following workaround:

(handler-bind ((warning #'muffle-warning))
  ...)

around the call to map-document to suppress the warnings and any related
performance issues.


Best regards
d.

Re: validating XML: document

Alexandre Rademaker — 2012-07-10T02:41:43

Hello All,

Four months ago I posted the following message. Does anyone have any
idea if the warnings was my mistake or a possible bug in the code of
CXML? In the second case, with some directions I would be glad to help
to fix the problem.

Best,
Alexandre

On Wed, Mar 14, 2012 at 10:06 PM, Alexandre Rademaker
<arademaker< at >gmail.com> wrote:

Inconsistency in creating %children vector

Raymond Wiker — 2012-07-09T18:39:24

I was scratching my head for a couple of hours today, because of an apparent failure of cxml-dom::make-space - it did not increase the size of the %children vector. On closer examination, it turned out that the reason was that the %children vector was not adjustable, at that point.

The only places in cxml-stp where %children is set up are in %unchecked-insert-child (parent-node.lisp) and normalize-text-nodes! in xpath.lisp. The first of these has (make-array ... :adjustable t ...), while the other does not. My guess is that normalize-text-nodes! might be called as part of an xpath evaluation, and if this happens in code that is actually building the cxml-stp structure, the result might be a situation where the %children vector is not adjustable.

I'm not sure if this is a valid use of cxml-stp/xpath, but there is at least one documented case where it has been done :-)

I could (trivially) change my code to not use xpath as part of the construction process, but adding :adjustable t to the call to make-array

Re: Real-world example: with attachment

Steven Nunez — 2012-06-11T08:04:38

Hi Ray,

Thanks. It does seems easier with xpath. Perhaps it's my stubbornness that keeps me hammering away at cxml-stp. If no one comes up with a good example for STP, I think I'll take the path of least resistance and do as you suggest.

Cheers,
- Steve


________________________________
 From: Raymond Wiker <rwiker< at >gmail.com>
To: Steven Nunez <steve_nunez< at >yahoo.com> 
Cc: "cxml-devel< at >common-lisp.net" <cxml-devel< at >common-lisp.net> 
Sent: Monday, June 11, 2012 5:14 PM
Subject: Re: [cxml-devel] Real-world example: with attachment
 

On Jun 11, 2012, at 05:50 , Steven Nunez wrote:
(defun algorithm-name (model)

I'd do something like the following:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defun algorithm-name (model)
  (xpath:with-namespaces ((nil "http://www.dmg.org/PMML-4_0"))
    (xpath:string-value (xpath:evaluate "//RegressionModel/< at >algorithmName" model))))

(defun model-coefficient-list (model)
  (xpath:with-namespaces ((nil "http://www.dmg.org/PMML-4_0"))
    (let ((regression-model-node (xpath:first-node (

Re: Real-world example: with attachment

Raymond Wiker — 2012-06-11T07:14:32

I'd do something like the following:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defun algorithm-name (model)
  (xpath:with-namespaces ((nil "http://www.dmg.org/PMML-4_0"))
    (xpath:string-value (xpath:evaluate "//RegressionModel/< at >algorithmName" model))))

(defun model-coefficient-list (model)
  (xpath:with-namespaces ((nil "http://www.dmg.org/PMML-4_0"))
    (let ((regression-model-node (xpath:first-node (xpath:evaluate "//RegressionModel" model)))
          (*read-eval* nil))
      `(,(xpath:string-value (xpath:evaluate "< at >modelName" regression-model-node))
        ,< at >(xpath:map-node-set->list
           (lambda (node)
             (list
              (xpath:string-value (xpath:evaluate "< at >name" node))
              ;; note: xpath:number-value may work here, depending on
              ;; your implementation... does not work with Lispworks
              ;; 6.1 under Mac OS X
              (read-from-string (xpath:string-value (xpath:evaluate "< at >coefficient" node)))))
           (xpath:evaluate "RegressionTable/NumericP

Real-world example: with attachment

Steven Nunez — 2012-06-11T03:50:18

Greetings again,

Perhaps a good example would be a real one. I'm starting with some simple functions, such as obtaining the model name, algorithm, etc, from the attached XML example. Can anyone provide an exemplary method for doing this? Below is my naive attempt, which are all failing miserably:

(defun algorithm-name (model)
  "Returns the algorithm for the model"
  (stp:filter-recursively (stp:of-name "algorithmName" (stp:namespace-uri model)) model))

I know this one will (if it were working) return a list, but since I can't get past the namespace problem haven't bothered to go any further. That function, I assume, is a rather trivial one for someone with some good examples or knowledge to work from. Here's a slightly more difficult function:

(defun model-coefficient-list (model)
  "Returns a nested list of the following structure: (<model-name>-coefficients (coefficient1 coefficient1-value) (coefficient1 coefficient2-value) ... (coefficientn coefficientn-value))"
  ...)

Something like this would

Real-world Example

Steven Nunez — 2012-06-11T02:46:23

Greetings again,

Perhaps a good example would be a real one. I'm starting with some simple functions, such as obtaining the model name, algorithm, etc, from the attached XML example. Can anyone provide an exemplary method for doing this? Below is my naive attempt, which are all failing miserably:

(defun algorithm-name (model)
  "Returns the algorithm for the model"
  (stp:filter-recursively (stp:of-name "algorithmName" (stp:namespace-uri model)) model))

I know this one will (if it were working) return a list, but since I can't get past the namespace problem haven't bothered to go any further. That function, I assume, is a rather trivial one for someone with some good examples or knowledge to work from. Here's a slightly more difficult function:

(defun model-coefficient-list (model)
  "Returns a nested list of the following structure: (<model-name>-coefficients (coefficient1 coefficient1-value) (coefficient1 coefficient2-value) ... (coefficientn coefficientn-value))"
  ...)

Something like this would

namespace-uri usage

Steven Nunez — 2012-06-11T02:27:54

Gents,

Can anyone tell me why namespace-uri can't give me the namespace for this test file?

<PMML version="4.0" xsi:schemaLocation="http://www.dmg.org/PMML-4_0 http://www.dmg.org/v4-0/pmml-4-0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.dmg.org/PMML-4_0">
  <Header copyright="Steve Nunez" description="Sample for Namespace">
    <Timestamp>2012-06-11T08:17:10.8</Timestamp>
  </Header>
</PMML>

* (defparameter *ns-test*
     (cxml:parse #p"ns-test.xml" (stp:make-builder)))

* (stp:namespace-uri *ns-test*)


This, and a few variations, give "There is no applicable method for the generic function". 

I suspect the answer is because I'm asking for the namespace of a document, whereas STP wants it for an element. That's fine, but different enough from the typical ways of XML processing to make figuring out how to use it rather difficult.

Does anyone have any working examples of processing documents typically found 'in the wild'? I've read the test.lisp file, but it simp

Re: cxml-stp usage

Raymond Wiker — 2012-06-10T17:42:41

One easy way might be to use xuriella-xpath, which has xpath:with-namespaces - not sure if this is possible in your case.

Re: cxml-stp usage

Steven Nunez — 2012-06-10T11:06:35

Hi David,

Right, your explanation makes it clear. What's the convention/best practice/idiom for picking out the namespaces? Would be a bit repetitious to keep typing that out.

Regards,
- Steve



________________________________
 From: David Lichteblau <david< at >lichteblau.com>
To: "cxml-devel< at >common-lisp.net" <cxml-devel< at >common-lisp.net> 
Sent: Sunday, June 10, 2012 8:49 PM
Subject: Re: [cxml-devel] cxml-stp usage
 
Hi,

Quoting Steven Nunez (steve_nunez< at >yahoo.com):

The call to OF-NAME is the problem, it needs to be
  (stp:of-name "RegressionModel" "http://www.dmg.org/PMML-4_0")
because:


your element is in the http://www.dmg.org/PMML-4_0 namespace, whereas you
specified the empty namespace.

Roughly speaking,
  local-name == SYMBOL-NAME
  namespace-uri == SYMBOL-PACKAGE

I.e. just like distinct symbols in different packages have got nothing
to do with each other just because they happen to have the same
SYMBOL-NAME in Lisp, similarly in XML, two elements with the same
local-name but a different namespa

Re: cxml-stp usage

David Lichteblau — 2012-06-10T10:49:12

Hi,

Quoting Steven Nunez (steve_nunez< at >yahoo.com):

The call to OF-NAME is the problem, it needs to be
  (stp:of-name "RegressionModel" "http://www.dmg.org/PMML-4_0")
because:


your element is in the http://www.dmg.org/PMML-4_0 namespace, whereas you
specified the empty namespace.

Roughly speaking,
  local-name == SYMBOL-NAME
  namespace-uri == SYMBOL-PACKAGE

I.e. just like distinct symbols in different packages have got nothing
to do with each other just because they happen to have the same
SYMBOL-NAME in Lisp, similarly in XML, two elements with the same
local-name but a different namespace are considered distinct.

[...]

An element with that local-name, right?


I think STP was pretty much finished when it was first written, because
it is basically a Common Lisp adaptation of XOM, so the design was clear
from the start.


I see no reason why you would have to call stp:first-child.  The example
works without that call.


d.

cxml-stp usage

Steven Nunez — 2012-06-10T07:51:34

Greetings all,

I'm a bit stumped by some of the STP functions and usage. The documentation is, well, a bit sparse, so it's not clear whether there really is a bug or I'm mis-interpreting what I'm reading. Here is a simple function that should return all the child nodes of an element with the local-name of 'RegressionModel':

(defun function-name (document)
  "Return the PMML model type of the document"
  (stp:filter-recursively (stp:of-name "RegressionModel") (stp:first-child document)))

According to the documentation of filter-recursively:
Return a list of descendant nodes of node in pre-order, from which nodes that do not satisfy predicate have been removed.and that for of-name:
This function creates a test function for nodes of this name.
and here's the local snippet of the xmls that I'm trying to filter for:

#.(CXML-STP:ELEMENT

Is stp stable enough for production use? It's been a while since it's been updated. I'm new to XML with LISP, so if there's a better documented or easier way to process

Exported symbols [was: Re: Possible bug in cxml-stp]

Raymond Wiker — 2012-05-25T10:28:59


[Subject: changed, as it was getting embarassingly inappropriate.]

I ended up modifying my add-element function, based on the code that you
included earlier. It now looks like this:

(defun add-element (parent element-name attributes &optional element-value)
  (let ((element-name (if (listp element-name) (first element-name)
element-name))
        (namespace-uri (if (listp element-name) (second element-name)
                          (cxml-stp:find-namespace (cxml::split-qname
element-name) parent))))
    (let ((new-element (cxml-stp:make-element element-name namespace-uri)))
      (cxml-stp:append-child parent new-element)
      (loop for (name value) on attributes by #'cddr
            do (cxml-stp:add-attribute
                new-element
                (cxml-stp:make-attribute
                 (princ-to-string value)
                 name
                 (cxml-stp-impl::find-attribute-namespace
                  (cxml::split-qname name) new-element))))
      (when element-value
        (cxml-stp:app

Any Clever Ways to Build an XForm from an XSD?

Steven Nunez — 2012-05-23T07:40:37

Greetings all,

Having been away from the XML world for a while, I'm back on a fairly large project using it. While travelling down one of the paths in the rabbit warren, I came across a need to generate a from (probably an XForm) from an XSD. The use case is the specification of data models via the XSD, and quickly generating a form that a human user could use to input data.

It seems that there's *almost* some pipeline of chtml/cxml/cxml-stp that would get us most of the way there, but I haven't found a magic incantation that will do the trick.

Does anyone know of any clever way to go from an XSD to a X/HTML form? It doesn't have to look pretty, just be functional. I do know there will be some things that can't be easily done; that's fine, I expect manual work to reach the final product. I'm just looking for a tool that will get us most of the way there, since we have a large number of input formats. Doesn't have to even be a LISP solution, though that would be much preferred.

Cheers,
    - Steve

Re: Possible bug in cxml-stp

David Lichteblau — 2012-05-22T22:02:04

Quoting Raymond Wiker (rwiker< at >gmail.com):

Possibly...  cxml currently has such a macro only for its serialization
API, i.e.

CL-USER> (cxml:with-xml-output (stp:make-builder)
           (cxml:with-namespace ("a" "http://a")
             (cxml:with-element "a:b"
               (cxml:text "xyz"))))
#.(CXML-STP-IMPL::DOCUMENT
   :CHILDREN '(#.(CXML-STP:ELEMENT
                  #| :PARENT of type DOCUMENT |#
                  :CHILDREN '(#.(CXML-STP:TEXT
                                 #| :PARENT of type ELEMENT |#
                                 :DATA
                                 "xyz"))
                  :LOCAL-NAME "b"
                  :NAMESPACE-PREFIX "a"
                  :NAMESPACE-URI "http://a")))

That API can't really modify STP in place though; the builder only knows
how to set up a full document.


d.