gmane.text.docutils.devel

Re: syntax highlighting directive

Mark Nodine — 2008-10-08T02:32:22

> David Goodger python.org> schrieb: > >> A syntax highlighting directive is a common request, and there are >> several implementations out there. Let's work on generalizing the >> directive for all of Docutils (so it works with all writers), and move >> it into the parser code. > > May I recall our earlier discussion on this topic (and some new thoughts > after the math-mode directive plans)? > > For the full proposal see syntax-highlight_ > (Newest version on SVN: syntax-highlight.txt_.) > > > `code-block` directive proposal > ------------------------------- > > Syntax > """""" > > :Directive Type: "code-block" > :Doctree Element: literal_block > :Directive Arguments: One (`language`) or more (class names), optional. > :Directive Options: None. > :Directive Content: Becomes the body of the literal block. > > The "code-block" directive constructs a literal block where the content is > parsed as source code and syntax highlight rules for `language` are > applied. If syntax rules for `language` are not known to docutils, it > is rendered like an ordinary literal block. > > For example, the following snipped is parsed and marked up as python source > code. The actual rendering depends on the style-sheet. :: > > .. code-block:: python > > def my_function(): > "just a test" > print 8/2 > > If the language argument is missing, a default language should be used. > Additional arguments might be used and e.g. passed to the parser > or (as class arguments) the output document. > > > Implementation > """""""""""""" > > Alternatives > '''''''''''' > > There are 2 alternatives for the implementation of a code-block directive: > > hight level markup > modeled on the ideas for the planned math-mode directive: > > * Store the content as-is in the doctree and tag it as source code of > lanugage `language` (i.e. store a literal-block element with > "code-block" and "`language`" classes). > > * Write according to the writers possibilities and user settings > (parse and mark-up by the writer (or backend) or render as > monospaced). > > low level markup: > result of the past discussion for the planned source-code directive: > > * Parse the content while reading and store as "rich" literal block. > > * Write similar to a "parsed-literal" block. Styling via style-sheet > is possible. > > > I invite all interested Docutils developers to participate in the > development of the code-block-directive_ Sandbox project. > > .. _syntax-highlight: > http://docutils.sourceforge.net/sandbox/code-block-directive/docs/syntax-highlight.html > > .. _syntax-highlight.txt: > http://svn.berlios.de/viewcvs/docutils/trunk/sandbox/code-block-directive/docs/syntax-highlight.txt > > .. _code-bloc-directive: > http://svn.berlios.de/viewcvs/docutils/trunk/sandbox/code-block-directive/ > Just for your info, the Perl reStructuredText has a syntax highlighting directive already, and it has the same name and syntax. Regarding the two levels of markup, it really does both. Without any options, it returns basically a parsed-literal block. If you give it the :color: option, it produces the high-level markup, which in the HTML writer at least, sets the span styles according to the syntax categories. As an implementation detail, it currently uses the Unix states program to do the syntax interpretation and fails gracefully (i.e., just returns a literal block) if the program does not exist. It has a few other options that you might want to consider. Check out the details at http://www.prest.de/doc/directives.html#including-a-marked-up-code-block Enjoy! --Mark ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

syntax highlighting directive (was: Adding rst2odt/odtwriter to the queue for the next release)

Guenter Milde — 2008-10-07T12:19:26

David Goodger python.org> schrieb: May I recall our earlier discussion on this topic (and some new thoughts after the math-mode directive plans)? For the full proposal see syntax-highlight_ (Newest version on SVN: syntax-highlight.txt_.) `code-block` directive proposal ------------------------------- Syntax """""" :Directive Type: "code-block" :Doctree Element: literal_block :Directive Arguments: One (`language`) or more (class names), optional. :Directive Options: None. :Directive Content: Becomes the body of the literal block. The "code-block" directive constructs a literal block where the content is parsed as source code and syntax highlight rules for `language` are applied. If syntax rules for `language` are not known to docutils, it is rendered like an ordinary literal block. For example, the following snipped is parsed and marked up as python source code. The actual rendering depends on the style-sheet. :: .. code-block:: python def my_function(): "just a test" print 8/2 If the language argument is missing, a default language should be used. Additional arguments might be used and e.g. passed to the parser or (as class arguments) the output document. Implementation """""""""""""" Alternatives '''''''''''' There are 2 alternatives for the implementation of a code-block directive: hight level markup modeled on the ideas for the planned math-mode directive: * Store the content as-is in the doctree and tag it as source code of lanugage `language` (i.e. store a literal-block element with "code-block" and "`language`" classes). * Write according to the writers possibilities and user settings (parse and mark-up by the writer (or backend) or render as monospaced). low level markup: result of the past discussion for the planned source-code directive: * Parse the content while reading and store as "rich" literal block. * Write similar to a "parsed-literal" block. Styling via style-sheet is possible. I invite all interested Docutils developers to participate in the development of the code-block-directive_ Sandbox project. .. _syntax-highlight: http://docutils.sourceforge.net/sandbox/code-block-directive/docs/syntax-highlight.html .. _syntax-highlight.txt: http://svn.berlios.de/viewcvs/docutils/trunk/sandbox/code-block-directive/docs/syntax-highlight.txt .. _code-bloc-directive: http://svn.berlios.de/viewcvs/docutils/trunk/sandbox/code-block-directive/ ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: Adding rst2odt/odtwriter to the queue forthe next release

Stefan Merten — 2008-10-06T07:08:45

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Dave, David and all! Last week (7 days ago) Dave Kuhlman wrote: +1 Count me in please :-) . I'm using (and patching) rst2odt for some time now and it's simply great to be able to convert reST to ODT and via OOo to Word. Grüße Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Processed by Mailcrypt 3.5.7 iQCVAwUBSOm5fQnTZgC3zSk5AQKEdAP9HOtt3qQ+K5+u0tMc238wU+0DUaXWt+gG hJ9vWR9LuQSAxKCk6sg7zGqiqH+ds0ocI7UwYJE/nuSfkSZJomSN2YgoQrm+P/V4 suk5XhCzN5es7DVr5UZS2z0HO8wM/9xUrVT8DfTdvC+LWCpsnTz62jFXoKfbYjYD 4xi1ggexXmo= =l6n/ -----END PGP SIGNATURE----- ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: Fw: Creating a file-like object.

David Goodger — 2008-10-06T14:38:38

On Mon, Oct 6, 2008 at 10:14, Sebastian Noack googlemail.com> wrote: I think I understand what you're asking now: you want lazy evaluation, or pipe-like execution (e.g. a generator). Short answer: no, Docutils doesn't support that. It can't really, since it takes multiple passes over the document -- it parses the input, then applies several transforms, then converts to the output format desired. A generator solution would only benefit the final stage, and the entire state would have to be kept in memory. Wrapping the output in a StringIO object (small) allows you to discard all the Docutils internals (many times larger). Use StringIO.

Fw: Creating a file-like object.

Sebastian Noack — 2008-10-06T14:14:54

------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: Creating a file-like object.

David Goodger — 2008-10-05T04:10:22

On Mon, Sep 29, 2008 at 12:13, Sebastian Noack googlemail.com> wrote: I don't understand what you're asking for. Please explain further, and include examples of what you'd like (input & output, code samples, etc.).

Creating a file-like object.

Sebastian Noack — 2008-09-29T16:13:07

Re: Adding rst2odt/odtwriter to the queue forthe next release

David Goodger — 2008-10-02T14:16:21

Those are actually a bunch of unique tests, not applicable to all writers. The ODT writer may need some of those (e.g. to test for the contents of the zip files, to test settings, etc.). But the main tests a writer needs are functional tests, which are described here: http://docutils.sourceforge.net/docs/dev/testing.html#functional-tests They run the entire Docutils system (in various configurations, each defined in a file in tests/functional/tests/) on input files (in tests/functional/input/) and compare the resulting output to a pre-defined expected output files (in tests/functional/expected). Yes. And the functional test framework (tests/test_funcitonal.py) will need to be modified to provide useful results for ODT's compound zip files. Everybody has a slightly different style. It may seem nit-picky, but adhering to the standard makes code easier for everyone to read. Some areas that I notice right away: * Vertical whitespace: one blank line between functions, two between classes. Don't have multiple statements on one line (e.g. ListLevel.get_sibling etc.). * Backslashes: use parentheses instead. * Use "raise Exception('message')" instead of the older form "raise Exception, 'message'". * Line length: keep lines under 80 columns (the line wrapping throws off the "code shape" sense). They were supported in a different way. The new API is class-based. The old one was function-based. 1. Yes, there's no need to check the API for directives any more. Better to track trunk. 2. There shouldn't be any directive code in a writer though. Directives are for the reST parser only. A syntax highlighting directive is a common request, and there are several implementations out there. Let's work on generalizing the directive for all of Docutils (so it works with all writers), and move it into the parser code.

Re: Adding rst2odt/odtwriter to the queue forthe next release

Dave Kuhlman — 2008-10-02T00:02:54

I'll try to follow that guidance on check-ins. I'm looking at the files in docutils/test/test_writers. I'll try to mimic one of those. The content we'd need to check is in content.xml inside that zipped .odt file. That XML is all on one line. So, I'll look into unzipping and then pretty printing it (perhaps with minidom). That would give the test something reasonable to compare against. OK. I've removed most of the commented-out stuff. About non-standard code, I've tried to follow PEP 8 ("Style Guide for Python Code") as much as I can. I've also looked at code in the Docutils implementation, and the code in odtwriter seems consistent with that. Can you think of anything in particular that I should look for? A hint would help. It's hard to see the flaws in your own baby. One of those conditionally defined classes was there because directives were not supported in Docutils prior to 0.4, I believe. I don't believe that we need to worry about that now. So, I removed the "if" statement. I'll also look at the others. Thanks for the guidance. I'll be back when I've made a bit more progress. - Dave -- Dave Kuhlman http://www.rexx.com/~dkuhlman ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: Adding rst2odt/odtwriter to the queue forthe next release

David Goodger — 2008-10-01T13:36:39

Yes, please!

Re: Adding rst2odt/odtwriter to the queue forthe next release

Guenter Milde — 2008-10-01T11:58:10

David Goodger python.org> schrieb: I see Python version 2.1 to 2.3 mentioned there. As of docutils 0.5, support for 2.1 was dropped. Testing with 2.4 and 2.5 seems reasonable nowadays too. Should I update the doc? Günter ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Docutils-develop mailing list Docutils-develop< at >lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/docutils-develop Please use "Reply All" to reply to the list.

Re: Adding rst2odt/odtwriter to the queue forthe next release

David Goodger — 2008-10-01T04:09:40

We do have documentation for this, at http://docutils.sourceforge.net/docs/dev/policies.html#check-ins No, it's not required that a writer be finished, just that it be reasonably complete: """ Reasonably complete means that the code must handle all input. Here "handle" means that no input can cause the code to fail (cause an exception, or silently and incorrectly produce nothing). "Reasonably complete" does not mean "finished" (no work left to be done). For example, a writer must handle every standard element from the Docutils document model; for unimplemented elements, it must at the very least warn that "Output for element X is not yet implemented in writer Y". """ Of course, completeness ("finished") is a desirable goal. Once a writer is in the core, that ought to happen. The writer should have at least a functional test to catch regressions. The ODT writer may need a test fixture or special processing to generate a useful result, since two zip files differing is not very useful -- the test should say *how* they differ. See http://docutils.sourceforge.net/docs/dev/testing.html Minor nits: * A quick look at the code reveals lots of commented-out junk code, and lots of code in a non-standard style. It could use a cleanup. * The conditional classes seem kludgey to me; why not break them out into another module, and import that conditionally?

Adding rst2odt/odtwriter to the queue for thenext release

Dave Kuhlman — 2008-09-29T17:52:00

I'd like rst2odt/odtwriter to be considered for the the next release of Docutils. Currently, when I run rst2odt.py on demo.txt, it successfully runs; it produces an output file that looks good to me; and it shows me that rst2odt does not support citations and sidebars. Are these required for inclusion? I suppose that I am asking how we decide whether to include it. I suspect that there needs to be some demand for it and that it needs to pass certain tests. I believe that David (G.) said that the ability to process demo.txt without failing was a requirement, so it has passed that test. So, is there anything I can do to move this forward or at least have it considered? And, by the way, we will have the same questions with rst2pdf, which I'm also hoping will be included in the Docutils distribution, soon. - Dave ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: I was thinking that reST would be just thething for this application...

David Goodger — 2008-09-22T17:19:02

Martin, I have no idea what you're asking for, beyond some hand-holding with transforms. There is some documentation available: http://docutils.sourceforge.net/docs/api/publisher.html (search for publish_doctree) http://docutils.sourceforge.net/docs/peps/pep-0258.html http://docutils.sourceforge.net/docs/dev/hacking.html http://docutils.sourceforge.net/docs/ref/doctree.html http://docutils.sourceforge.net/docs/ref/docutils.dtd and the code itself Yes, it's inadequate. Sorry, but nobody has had the right combination of time/will/need to remedy that. Contributions are welcome. You'd do well to ask concrete questions, minimize the flowery prose, and avoid disparaging remarks directed at the very people you're asking for help. Care to try again?

I was thinking that reST would be just the thingfor thisapplication...

Martin Maney — 2008-09-22T15:09:42

How to describe it? I need a not too user-unfriendly format for entering items that are mostly a scrap of descriptive text and a hypertext link. The tricky bit is that this is part of a system that has a catalog of URLs, and I want to allow items to refere to cataloged links by name - so that when a URL changes, there's one place to fix it, or to redirect to a "sorry, this resource is no longer available" (and, in my dreams, maybe suggest alternatives... but that's only a dream for now). I've been trying to avoid reinventing this wheel, but the first few things I looked at didn't do well on the user ratings. reST's hyperlink syntax has always seemed kind of funky to me, but my users seem to like it. The problem then would be that this seems to call for a transform, and it's probably quite simple... except that the documentation of that area is so incomplete as to be useless. I've groveled the code enough to decide that if I have to go that route it truly will be simpler to reinvent the wheel once again (this so is a Pythonic trope, isn't it?). I'd lose the other things that reST provides, but frankly they're not of great importance for items, at least we currently envision using them. So sorry, but code is really crappy documentation, even when the trees aren't lost in an ocean of do-nothing classes (was this designed by someone with extensive Jave experience, perhaps?). From what I've been able to make out, dimly, I believe I need to 1) get the item parsed into nodes - that turns out to be easier than the __doc__ suggested, just publish_doctree(item_string) (okay, with stderr redirected if my extended syntax isn't quite legal reST form). 2) I think I can see how to walk the tree - this might actually be easiest if I do use an invalid syntax, since then I can ignore everything but problematic nodes, and I then get the source text from their single Text child. Here I encounter a problem - how the heck do I construct a proper reference node to replace the problematical? After tracing all the twisted class inheritance, all I see in the constructor is the raw source, a text item, and the who knows what they should/can hold *children and **attributes. ...or is that not a reasonable way to get there? I did think at first that it might be handled during parsing, but that part of the process seemed even less well-lit. Okay, that's enough questions for now - I'm already basing most of this on guesswork in lieu of documentation. All suggestions, pointers to more obscure docs, examples of similar work appreciated. And contrary to my usual preferences, please do CC me on replies as I'm not hooked up to the list. Thanks!

Re: Include roman.py as a submodule?

Alan G Isaac — 2008-09-12T19:12:56

On 9/12/2008 3:07 PM Robert Kern apparently wrote: However if you can be specific about the "problems" you encounter, it may prove possible to change his mind. Cheers, Alan Isaac ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: Include roman.py as a submodule?

Robert Kern — 2008-09-12T19:07:07

The last time this was suggested, David Goodger did not want to do so since docutils is public domain whereas roman.py has a copyright and furthermore that he did not want to ask the author to release it into the public domain. See the thread "roman.py in site-packages" in docutils-user on 2008-07-22.

Include roman.py as a submodule?

Georg Brandl — 2008-09-12T17:20:54

Just a suggestion for the next release: why not include roman.py as a submodule of docutils? There are lots of problems people have with the status quo, where roman is distributed and installed as a separate toplevel module... Georg

[ docutils-Support Requests-2100641 ] not oninternap mirror (San Jose)

SourceForge.net — 2008-09-08T17:23:38

Support Requests item #2100641, was opened at 2008-09-08 10:23 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422031&aid=2100641&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Install Problem (example) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nathan Youngman (nyoungman) Assigned to: Nobody/Anonymous (nobody) Summary: not on internap mirror (San Jose) Initial Comment: The default mirror SF picks for me is internap in San Jose. Seems the 0.5.0 download isn't there, and gives an error. I originally got this error while trying to easy_install Sphinx. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422031&aid=2100641&group_id=38414 ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: Patch [ 1878977 ] make_id(): deaccentcharacters

David Goodger — 2008-09-04T20:37:23

I uploaded an updated patch that handles Python 2.2 (disables the feature by catching exceptions). When we drop 2.2-compatibility, the try/except can be removed. Add some tests (the more comprehensive, the better) and it's good to go. https://sourceforge.net/tracker/index.php?func=detail&aid=1878977&group_id=38414&atid=422032

Re: Patch [ 1878977 ] make_id(): deaccentcharacters

engelbert gruber — 2008-09-04T06:48:22

i uploaded a patch using unicodedata.normalize this reduces the translate dictionary to 41 entries unicodedata.normalize does not exist in python2.2 possibly unicodedata.decomposition does similar but mapping from ``\u00df`` to ``sz`` is not supprted in python2.2 string.translate either. https://sourceforge.net/tracker/index.php?func=detail&aid=1878977&group_id=38414&atid=422032 ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/