gmane.text.yaml.general

Re: Checksums

Oren Ben-Kiki — 2013-04-29T17:41:54

I wouldn't use %CRC for this, YAML doesn't really go for user-defined
directives. One can easily just use a special document:

---
crc: ...
---
a: foo
...

Or some other valid YAML solution.


On Mon, Apr 29, 2013 at 5:30 PM, I Heart Robotics
<iheartrobotics-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>wrote:

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr_______________________________________________
Yaml-core mailing list
Yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/yaml-core

Checksums

I Heart Robotics — 2013-04-29T14:30:40

|Is there a better way of passing checksums of streams?
The use case is sending YAML data over a serial connection that occasionally gets errors or dropped characters.

%YAML 1.2
%CRC B8C5938E
---
a: foo
b: bar
c: baz
...
%YAML 1.2
%CRC 498BBC05
---
a: alpha
b: beta
c: gamma
...|


------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr

Re: PyYAML hangs when reading a large YAML file

Misha Penkov — 2013-04-18T12:55:33


Thank you for your suggestion.  I noticed that RAM usage grew slowly until
it consumed almost the entire available RAM (64GB) on my machine.

Why is this happening, and how can I get it to load the YAML file properly?
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter_______________________________________________
Yaml-core mailing list
Yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/yaml-core

Re: PyYAML hangs when reading a large YAML file

Alexey Zakhlestin — 2013-04-18T12:11:55

On 18.04.2013, at 15:17, Misha Penkov <misha.penkov-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:


did you check resource-usage? could it be, that Python just grabbed all RAM it could reach and started to actively use swap?


------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

PyYAML hangs when reading a large YAML file

Misha Penkov — 2013-04-18T11:17:36

Hi,

I'm trying to load data from a large YAML file (approx. 300MB).  PyYAML
hangs when I try to load using this code:

    import yaml
    y = yaml.load(open("/tmp/tmp6aJfKz"))

The YAML file itself is valid (it is being output by OpenCV).

Can anyone suggest a way to diagnose this problem?  I can provide the file,
but as I've already mentioned, it's fairly large.

Cheers,
Michael
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter_______________________________________________
Yaml-core mailing list
Yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/yaml-core

Re: Top level block scalar without indentation

KOSEKI Kengo — 2013-04-14T04:30:08

Hi Oren,

Thanks for the clarification! I created a ticket for this.

  http://pyyaml.org/ticket/288



Is YAML 1.1 same?

In YAML 1.1 spec, the rule [117] max(-1, 0) set n = 0.
So I think [166] detect(m) must return 0, not 1..

  [117] c-l+block-scalar(n) ::= c-l+folded(max(n,0)) | c-l+literal(max(n,0))
  [175] c-l+literal(n) ::= c-b-block-header(literal,m,t)
                                    l-literal-content(n+m,t)

I'm not sure I'm understanding these parametered production rules..

Anyway, I hope I can use top level block scalar with libyaml.


2013/4/14 Oren Ben-Kiki <oren-vmbulFz3td5g9hUCZPvPmw< at >public.gmane.org>:

Re: Top level block scalar without indentation

Oren Ben-Kiki — 2013-04-13T16:25:31

Well, example 9.5 is pretty explicit in what the spec says. Note that the
auto-detect does not return 0, it returns at least 1, but since we start
with -1 this results with an overall 0 indentation (since production 207
uses -1 and not 0).

The same logic holds for both of your samples; example 9.3 shows an
unindented plain at the top level.

Oren.



On Sat, Apr 13, 2013 at 2:10 PM, KOSEKI Kengo <koseki-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter_______________________________________________
Yaml-core mailing list
Yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourc

Top level block scalar without indentation

KOSEKI Kengo — 2013-04-13T11:10:31

Hi,

I noticed that LibYAML doesn't parse block scalar without indentation.

  Sample: 1
  ---
  XXX
  YYY
  ZZZ

  Sample: 2
  --- |
  XXX
  YYY
  ZZZ

I expected that these samples would be parsed like this:

  [{"Sample": 1}, "XXX YYY ZZZ"]
  [{"Sample": 2}, "XXX\nYYY\nZZZ\n"]

LibYAML can parse Sample 1 but not Sample 2.

  $ ./run-parser Sample1.yaml
  [1] Parsing 'Sample1.yaml': SUCCESS (11 events)

  $ ./run-parser Sample2.yaml
  [1] Parsing 'Sample2.yaml': FAILURE (10 events)

PyYAML returned error.

  expected '<document start>', but found '<scalar>'


I checked some implementations. The results are:

  Syck (YAML 1.0) - Ruby
    Sample1 ... PASS
    Sample2 ... PASS

  YAML.pm (YAML 1.0) - Perl
    Sample1 ... FAIL
    Sample2 ... PASS

  PyYAML (YAML 1.1) - Python / LibYAML
    Sample1 ... PASS
    Sample2 ... FAIL

  Psych     (YAML 1.1) - Ruby / LibYAML
    Sample1 ... PASS
    Sample2 ... FAIL

  SnakeYAML (YAML 1.1) - Java / based on LibYAML
    Sample1 ... PASS
    Sample2 ... FAIL

  JS-YAML

Re: feature request for libyaml: support "\/" escape sequence for YAML 1.2/JSON compatibility

Ingy dot Net — 2013-04-02T04:37:10

That's up to Kirill. I just forked the libyaml svn repo onto github so that
the community could maintain it.

On Mon, Apr 1, 2013 at 4:46 PM, Patrick Pelletier <ppelletier-bXq66PvbRDbQT0dZR+AlfA< at >public.gmane.org>wrote:

------------------------------------------------------------------------------
Own the Future-Intel(R) Level Up Game Demo Contest 2013
Rise to greatness in Intel's independent game demo contest. Compete 
for recognition, cash, and the chance to get your game on Steam. 
$5K grand prize plus 10 genre and skill prizes. Submit your demo 
by 6/6/13. http://altfarm.mediaplex.com/ad/ck/12124-176961-30367-2_______________________________________________
Yaml-core mailing list
Yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/yaml-core

Re: feature request for libyaml: support "\/" escape sequence for YAML 1.2/JSON compatibility

Patrick Pelletier — 2013-04-01T23:46:31


Thanks!  That's awesome!


Cool, great to hear libyaml has migrated to git.  But perhaps the 
libyaml home page should be changed to mention the git repository?  It 
still only mentions subversion.

--Patrick


------------------------------------------------------------------------------
Own the Future-Intel&reg; Level Up Game Demo Contest 2013
Rise to greatness in Intel's independent game demo contest.
Compete for recognition, cash, and the chance to get your game 
on Steam. $5K grand prize plus 10 genre and skill prizes. 
Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d

Re: feature request for libyaml: support "\/" escape sequence for YAML 1.2/JSON compatibility

Ingy dot Net — 2013-03-31T15:09:40

Hi Patrick,

This was added to libyaml a couple months ago:

https://github.com/yaml/libyaml/commit/66bcb985569e7b643823c9ba1aeb47892583aeef

Note that it's on the yaml-1.2 branch. Also note that this new github repo
is the canonical source at this time.

Cheers, Ingy

On Wed, Mar 27, 2013 at 5:31 PM, Patrick Pelletier <ppelletier-bXq66PvbRDbQT0dZR+AlfA< at >public.gmane.org>wrote:

------------------------------------------------------------------------------
Own the Future-Intel(R) Level Up Game Demo Contest 2013
Rise to greatness in Intel's independent game demo contest. Compete 
for recognition, cash, and the chance to get your game on Steam. 
$5K grand prize plus 10 genre and skill prizes. Submit your demo 
by 6/6/13. http://altfarm.mediaplex.com/ad/ck/12124-176961-30367-2_______________________________________________
Yaml-core mailing list
Yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/yaml-core

feature request for libyaml: support "\/" escape sequence for YAML 1.2/JSON compatibility

Patrick Pelletier — 2013-03-28T00:31:24

I attempted to submit a feature request ticket for libyaml:

http://pyyaml.org/newticket?component=libyaml

but it rejected my ticket, telling me it was spam.  (I could go off on a 
whole rant about how so-called "spam filtering" makes it impossible for 
legitimate users to get anything done these days, but that would be 
off-topic.)

Anyway, since I can't submit a ticket, sending the feature request to 
this list seemed like the next-best thing.

Here is the text of the ticket I attempted to submit to libyaml:

YAML 1.2 adds support for the escape sequence "\/", which was not 
present in YAML 1.1:

http://www.yaml.org/spec/1.2/spec.html#id2776092

YAML 1.2 added this escape sequence in order to be compatible with JSON. 
  (Since YAML's goal is to be a superset of JSON.)

Although libyaml is only a YAML 1.1 parser, it would be nice to have 
this feature, and adding it shouldn't cause any trouble with parsing 
YAML 1.1.

This is all that's needed:

{{{
--- a/scanner.c~
+++ b/scanner.c
< at >< at > -3164,6 +3164,10 < at >< at > y

Re: Load Delegation

Trans — 2013-02-21T18:22:51

Shoot. "good practice" != "enforce", so I will have to deal with the
implications of the possibility in the parser. :-(

Hey, btw, could you change the default respond-to address of the mailing
list to "yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org"?  I try to remember to add it
every time I use respond, but I *always* forget eventually. It makes for a
very broken conversation for others trying to read on the list!!!



On Thu, Feb 21, 2013 at 1:08 PM, Oren Ben-Kiki <oren-vmbulFz3td5g9hUCZPvPmw< at >public.gmane.org> wrote:

Re: Load Delegation

Trans — 2013-02-21T16:47:10


Yes, that *should* be the case. But people fudge. Ruby's Psych
implementation recognizes appox. 13 different tags when one defines a
single "full" tag. Perfect round-tripping is of course not necessary, but I
thought it would be nice if it were at least smart enough to have `document
== YAML.dump(YAML.load(document))`, i.e. if you immediately emit what you
just loaded it will come out exactly the same, local tags and all. Or is
that a bad idea in itself? Should all local tags always become the "one
full tag" when emitted? Note, the same goes for decoration. If a !!str, for
instance, was loaded using the literal `|` form, it seems like a nice idea
to remember that and emit it the same way.




Less likely to occur in the same document, but it could. For instance, a
schema could translate a !!int to an Integer object, but also !number to an
Integer object. But Integer is singleton in Ruby, so there is only ever one
object for a given number. So if a document has:

   ---
   a: !!int 42
   b: !number 42

Then

Load Delegation

Trans — 2013-02-21T15:47:48

I'd like to get some feedback on an idea I have to implementing a YAML 
parser.

I was thinking that instead of instantiating the objects to native types 
directly, it could instantiate a delegator around the native type. e.g. 
instead of (in Ruby code):

     YAML.load('--- "string"') => "string"

It would produce

     YAML.load('--- "string"') => Y("string")

For all practical purposes Y("string") behaves just like "string".

The main reason for doing this is because of immutable types. Immutable 
types are difficult to load with self-referencing anchors --indeed the only 
solution I found was to forbid it. Immutable types also might not round 
trip well, b/c they are often singleton. So if the same object comes in via 
two different tags --say a schema supports both `!foo` and 
`foo.org,2000:foo`, it is only possible to emit it with on or the other. 
There is no way to remember which it came in with. Using the delegator is 
also nice b/c schemas then don't need to specify the type-class a tag goes 
with

Re: Tag Spec

Oren Ben-Kiki — 2013-02-03T20:02:37

Using `/` for indicating a global tag would go against writing stuff like
`!ruby/sym` (I hesitate to mention this given the latest issues with this
being a security loophole... but anyway).

The current rules were intentionally designed to promote the use of
suffixes. Using !<...> is intentionally ugly. You "should not", in general,
specify full URIs unless under special circumstances. The idea is that you
"should" use suffixes and isolate the full-URI prefixes in there. Whether
or not this intent is "correct", it is what it is.

Getting rid of `!<...>` and replacing it with a rule saying "is a complete
URI if it contains :, is a suffix otherwise" - hmmm. I'm not convinced, but
there might be something in it. Either way, this is be YAML 2 material.

Have fun,

Oren Ben-Kiki

On Sun, Feb 3, 2013 at 9:39 PM, Trans <transfire-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps f

Re: Tag Spec

Trans — 2013-02-03T19:39:38


:-) Indeed!

It is a matter of ambiguity. The !<verbatim> syntax was added very late in

The thinking isn't "domain" vs. "local", the thinking is "full arbitrary

Though in practice that's what people see. Perhaps the better terminology
is "globally unique" vs "local" tags. Although determining if a prefix is
applied would depend on whether the tag contains a `:` or not, sticking
with current spec, !<...> tags would still not resolve. To clarify the
difference:

    %TAG ! tag:foo.org/
    ---
    - !foo                   => !<tag:foo.org/foo>
    - !tag:bar.org/bar   => !<tag:bar.org/bar>

Also, I think this would open up `!<foo>` to be a legal local tag, rather
then the degenerate global tag it is now b/c it is not a valid URI. So,
without a `%TAG !` directive:

    ---
    - !foo                 => !<foo>

So the `!` would no longer have any significance if a tags *name*. It would
be used only to designate a tag and to sub prefixes, but a local tag would
not need to be `!foo` any more, just `foo`.

If w

Re: Tag Spec

Oren Ben-Kiki — 2013-02-03T18:31:07

The question isn't one of ease of parsing (we all know YAML isn't easy to
parse :-). It is a matter of ambiguity. The !<verbatim> syntax was added
very late in the game, to allow avoiding the tag prefix games.

The thinking isn't "domain" vs. "local", the thinking is "full arbitrary
URI" (inside <...>) vs. "suffix added to some arbitrary URI prefix"
(without the <...>). The difference between "!", "!!" and "!foo!" is just
"which URI prefix should we use here".

If we said "we don't attach any prefix if what follows the "!" looks like a
complete URI" we'd be entering a world of pain. URIs can be in all sort of
forms: "urn:isbn:0-395-36341-1" is a URI and hence (if someone wanted) !<
urn:isbn:0-395-36341-1> would be a valid (if somewhat insane) tag. So
"looking like a complete URI" isn't really easy to define. "Looking like a
tag URI" is well-defined, but YAML really doesn't insist on using "tag
URIs", even though we call the "node type tags", well, tags.

Have fun,

Oren Ben-Kiki



On Sun, Feb 3, 2013 at 8:0

Tag Spec

Trans — 2013-02-03T18:02:11

I was reading over the spec on Tags. There is an aspect of it that is
confusing. To the general viewer, it is not intuitive that the following
tag is not a "domain" tag:

    --- !tag:foo.org:org

It is actually a local tag with a very domain-esque name. The correct tag
is:

    --- !<tag:foo.org:org>

That's a very technical distinction and I think too difficult.

Would it be acceptable to just designate domain tags as any tag that
contains a `:` or `/`? I realize it might not be as efficient to parse, but
it would be a whole lot more comprehensible to humans.
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_jan_______________________________________________
Yaml-core mailing list
Yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/yaml-core

Re: Help understanfing tag resolution confusion

Trans — 2013-02-03T09:04:09

Ok. I think I understand. I also see now why it seems confusing. It's not
very intuitive that the initial `!` doesn't just mean "tag", it means
"local tag", unless that is if a `%TAG ! foo` directive has been used. Then
they are transformed into "domain tags" without any initial `!`.

In other words, when someone sees something like:

    --- !foo.org:person
    name: Tom

They think that's a domain tag b/c it resembles a domain tag. They do not
realize it needs to be.

    --- !<foo.org:person>
    name: Tom

In hindsight this was probably one of the arguments for using `!!` for
local tags instead of `!`.
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_jan_______________________________________________
Yaml-core mailing list
Yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/list

Re: Help understanfing tag resolution confusion

Trans — 2013-02-03T08:51:11


On Friday, February 1, 2013 2:36:38 PM UTC-5, Trans wrote:

Sigh, Oren has answered this, but it appears the google groups to 
mailing-list bridge is busted. I am so sick of things not working !!! 
Actually, it would be nice if we were able to use a github issues page for 
some questions, like this topic for instance. But if doesn't seem like 
anyone frequents those "forums" --or honestly which github project is the 
"right" project. But I'm thread jacking, I come back to this in another 
thread.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_jan_______________________________________________
Yaml-core mailing list
Yaml-core-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/yaml-core