gmane.os.cygwin.patches

Re: [PATCH] DocBook XML toolchain modernization

Corinna Vinschen — 2013-06-04T11:53:27

Hi Warren,

On May 23 21:50, Corinna Vinschen wrote:

Unfortunately it doesn't work. I just changed only the new-features.xml
file and `make' claimed "Nothing to be done for `all'".

Inspecting the generated Makefile.dep, I see this:

/home/corinna/src/cygwin/vanilla/winsup/doc/cygwin-ug-net//home/corinna/src/cygwin/vanilla/winsup/doc/cygwin-ug-net.html: legal.xml overview.xml setup-net.xml using.xml programming.xml
/home/corinna/src/cygwin/vanilla/winsup/doc/faq//home/corinna/src/cygwin/vanilla/winsup/doc/faq.html: faq-what.xml faq-setup.xml faq-resources.xml faq-using.xml faq-api.xml faq-programming.xml faq-copyright.xml

As you can see, the paths of the make targets are wrong, *and* the
list of dependencies is broken as well.

Running xidepend in the source file results in:

cygwin-ug-net/cygwin-ug-net.html: legal.xml overview.xml setup-net.xml using.xml programming.xml ov-ex-win.xml ov-ex-unix.xml highlights.xml new-features.xml setup-env.xml setup-maxmem.xml setup-locale.xml setup-files.xml pathnames.xml textbinary.xml filemodes.xml specialnames.xml cygwinenv.xml ntsec.xml cygserver.xml ../utils/utils.xml effectively.xml gcc.xml gdb.xml dll.xml windres.xml
faq/faq.html: faq-what.xml faq-setup.xml faq-resources.xml faq-using.xml faq-api.xml faq-programming.xml faq-copyright.xml

This looks a lot like you're still testing this with a in-tree build.

Again, as cgf already wrote, we do *NOT* support in-tree builds. Always
test with a separate build directory, please.

I fixed this now temporarily like this:

- I fixed the generation of the basename in xidepend to take VPATH
into account.

- I moved the `include Makefile.dep to the end of Makefile.in, otherwise
the Makefile.dep rules clobber the all rule.

- I dropped the "$(srcdir)" from that include since Makefile.dep is
created in the build dir.

- I fixed the Makefile.dep build rule to work from any build dir.

- I removed faq.xml as dependency from the Makefile.dep build rule
since the faq sources are given via $FAQ_SOURCES anyway.

- I added the necessary rule to rebuild Makefile automatically if
Makefile.in changes.

This works now, but it's unsatisfactory, because we now have three
different ways to implement dependencies:

- cygwin-api deps are generated by tweaking Makefile.in via doctool

- faq deps are generated via setting a variable and referring the
variable in the build dependency.

- cygwin-ug-net deps are generated via xidepend and create a separate
Makefile.dep file with a dependency rule.

It would be nice if this could be merged into one unified way to
generate the deps soonish, with the simple requirement that it should
work always, and from a spearate build dir in the first place.

Thanks,
Corinna

Re: [PATCH] DocBook XML toolchain modernization

Corinna Vinschen — 2013-05-23T19:50:00

That should be sufficient.  We're only adding files pretty seldom
anyway.


Thanks,
Corinna

Re: [PATCH] DocBook XML toolchain modernization

Warren Young — 2013-05-23T19:03:38

I noted that in the original proposal: one of the things you got from 
doctool is automatic dependency generation.  I'd put an item on the 
Wishlist to this effect, saying I needed to replace it.

At your prompting, I now have.  There is a new script called xidepend 
which generates Makefile.dep, which is included (and cleaned, if asked) 
by Makefile.in.

It's not perfect.  Because of the time it takes to run the dependency 
chaser, I've elected to make it dependent on only changes to the 
top-level XML files.  This means that if you add an XInclude to one of 
the leaf files, the referenced file won't get added to the dependency 
list for the top-level file that indirectly depends on it until you 
remove Makefile.dep, forcing its re-generation.


Yes, because it's still using doctool.

Re: [PATCH] DocBook XML toolchain modernization

Corinna Vinschen — 2013-05-23T14:11:40


Never mind cygwin-api.  This seems to work fine.  Just cygwin-ug-net
has this dependency problem.


Corinna

Re: [PATCH] DocBook XML toolchain modernization

Corinna Vinschen — 2013-05-23T14:02:11

Hi Warren,

On May  1 18:11, Warren Young wrote:

Note quite, as I just found while writing new documentation.  For some
reason doc/Makefile.in has lost all dependencies to the .xml source
files as far as cygwin-ug-net and cygwin-api is concerned.  There's a

  faq/faq.html : $(FAQ_SOURCES)

dependency but no such depedency for the other files.  So, if I just
built the docs, then change, say, pathnames.xml, or cygwinenv.xml,
another `make' will do nothing:

  $ make
  make: Nothing to be done for `all'.

Can you fix that, please?


Thanks,
Corinna

Re: [PATCH] DocBook XML toolchain modernization

Corinna Vinschen — 2013-05-13T14:26:33

I have not the faintest problem here, so I guess this means, just nuke
'em.


Corinna

Re: [PATCH] DocBook XML toolchain modernization

Corinna Vinschen — 2013-05-13T14:25:13

...and as long as we still have a writable FAQ ;)

What we have now looks good to me.  It builds fine on Linux outside
the source tree and the files, albeit calld *.xml now, still have this
cosy sgml feeling to them :)


Thanks,
Corinna

Re: May I remove setup.xml and cygwin-ug.xml?

Christopher Faylor — 2013-05-11T00:16:46

Sounds like they should go.  Removing them is not a permanent thing so
there is no harm in doing a 'cvs delete'.  They can always be resurrected
if needed.

cgf

May I remove setup.xml and cygwin-ug.xml?

Warren Young — 2013-05-10T18:19:49

These files in winsup/doc appear to have been replaced by setup-net.xml 
and cygwin-ug-net.xml.  The Makefile doesn't use either of these as 
input to any of its outputs, and they're not referenced by any of the 
other input files.

Further, if you try to build a document from either of these, you get 
errors due to missing XML chunks.  (e.g. setup-reg.xml, referenced by 
setup.xml, doesn't exist.)

When I first discovered this, I thought for sure it meant I'd somehow 
screwed up the SGML to DocBook XML conversion, but ruled that out thus:

     $ grep -Rsl setup-reg winsup

If you do that to both the current tree and to a tree rolled back to 
before I got my commit bit, you find that only setup.{xml,sgml} contains 
that string.  It means the pre-Warren tree couldn't build documents from 
this file, either.

I'm not in any particular hurry to get rid of these files.  They only 
trouble they're causing is the same any clutter causes.  I want to make 
sure no one knows a reason either has to remain in existence before I 
nuke them.

Re: [PATCH] DocBook XML toolchain modernization

Warren Young — 2013-05-06T21:21:24

Okay, that one line change is now checked in, with a ChangeLog entry.

Re: [PATCH] DocBook XML toolchain modernization

Christopher Faylor — 2013-05-04T05:27:46

I'm not in the middle of anything.  I checked in enough so that I can
once again build a snapshot.


I'm not planning on changing anything else.  If my change doesn't result
in usable documentation then I'd appreciate it if you would fix it.

cgf

Re: [PATCH] DocBook XML toolchain modernization

Warren Young — 2013-05-03T22:22:40

Sorry about that.

It looks like you're in the middle of fixing it, so rather than fight 
with you over the tree, I'll just let you continue.

I've got one element of the fix here that you haven't yet checked in, 
though.  You need to change the XInclude in cygwin-api.in.xml back to a 
DOCTOOL directive:

<               <xi:include href="legal.xml"/>
---
 > DOCTOOL-INSERT-legal

This is because xsltproc (via xmlto) doesn't know about $(srcdir) for 
XIncludes.  It looks in the same directory as the input XML and doesn't 
see legal.xml, and dies.  If you "#include" this via doctool, problem 
solved.

Bottom line, you can't mix doctool includes and XIncludes.

Re: [PATCH] DocBook XML toolchain modernization

Christopher Faylor — 2013-05-03T20:30:01

It looks like you broke the ability to build outside of the source tree.
We don't support building *in* the source tree so that's a pretty serious
breakage.

% make
make: *** No rule to make target `faq*.xml', needed by `faq/faq.html'.  Stop.

Re: [PATCH] DocBook XML toolchain modernization

Warren Young — 2013-05-02T00:14:34

I've created this.  There is a new output, winsup/doc/faq/faq.body, 
generated from faq.html whenever it changes.

The core of the script is only about 10 lines, as expected.  Comments, 
whitespace and error checking balloon that to 44 LOC.

Re: [PATCH] DocBook XML toolchain modernization

Warren Young — 2013-05-02T00:11:03

Applied.

It all seems to build the same as before, even after 'make clean'.

Re: [PATCH] DocBook XML toolchain modernization

Christopher Faylor — 2013-05-01T02:33:32

I guess I should have said: "Yes, I have seen Doxygen 'in my travels'"

Re: [PATCH] DocBook XML toolchain modernization

Warren Young — 2013-05-01T01:28:19

You might be mixing two of my proposals together.

This offer to provide a "FAQ body contents only" output would probably 
just be a ~10 line Perl script to extract the text between <body> and 
</body> in an input HTML file and store it in an output file, then tie 
the two together with a Makefile dependency.

The idea is that on your end, all you do is change the referent from 
.../winsup/doc/faq/faq.html to something like .../faq-body.html.

I have decided: the script shall be called bodysnatcher.pl. :)

Separately, I have proposed improving the styling of, e.g.:

http://cygwin.com/cygwin-ug-net/cygwin-ug-net.html

I would prefer not to copy and hack on your CSS file.  I'd be happier if 
I could just reference it via URL.[1]  Unfortunately, while the standard 
DocBook XSL stylesheets output HTML that is easy to style with CSS (lots 
of classes, well-named divs, etc.) its names are all different from 
those you have used in your HTML.

Later, we can talk about using bodysnatcher.pl more broadly, to make a 
version of the user guide that will pour into your new navbar HTML 
files.  At that point, we'll need to talk about a way to merge our two 
CSS variants.

By the way, the top page of cygwin.com has two different FAQ links.  The 
one that points to /faq/faq-nochunks.html should probably point to 
/faq.html.


*shrug*

It's a common practice[2] to have verbose comments on public interfaces, 
and to format them in a way to make reference documentation generation 
easy.  Doxygen knows one common syntax for this, and its output is 
beautiful and useful.

Here's a Doxygen based reference manual I created:

     http://tangentsoft.net/mysql++/doc/html/refman/

If you decide it's better to fully extract the API docs from the code, I 
can go with that instead.


I have no problem waiting on some of this stuff until then.  I'm not in 
any hurry.  I'm just asking questions as they occur to me so I don't get 
blocked later when I decide to start making changes.



[1] http://goo.gl/6U6gG
[2] https://en.wikipedia.org/wiki/Comparison_of_documentation_generators

Re: [PATCH] DocBook XML toolchain modernization

Christopher Faylor — 2013-05-01T00:31:54

Yeah, that's what I thought you'd say.  I don't think it's worth the
effort and expense of duplicating Cygwin's CSS elsewhere but maybe
there's a clever way to avoid the html nesting which wouldn't require
that.


I'd rather just move this out of the code entirely.  The user visible
interfaces aren't going to change and we haven't made a habit of
adding new DOCTOOL tags.  I don't know who first thought that adding
these was a good idea (it may predate my time on the project even
though CVS insists that I added it with version 1.1) but, if Corinna
agrees when she gets back, I'd like to just get rid of these.

cgf

Re: [PATCH] DocBook XML toolchain modernization

Warren Young — 2013-04-30T21:53:58

Conceivably browsers could stop tolerating it.

Given how tolerant browsers still are, that eventuality might seem 
unlikely, but such shifts have happened.

In the early web, this was considered acceptable HTML:

     <a name="foo">
     <h1>Foobie</h1>

In older browsers, this wouldn't do anything more than let you reference 
#foo within the document.

Today's browsers, though, see an open <a> tag and treat the entire rest 
of the page as one big hyperlink.  You can get weird effects, like 
paragraphs of text highlighting with the "clicked link" color when you 
click on such a page.  To fix it, you must close the tag, so:

    <a name="foo"/>
    <h1>Foobie</h1>

That, or change from named anchors to fragment identifiers:

     <h1 id="foo">Foobie</h1>

But I won't insist that you accept the new output. :)


The final removal of doctool requires replacing the DOCTOOL/SGML 
comments in winsup/cygwin/{path,pinfo}.cc with Doxygen comments, and 
folding most of the contents of winsup/cygwin/*.sgml into Doxygen 
comments within the relevant source files.

I haven't deeply looked into this.  The only thing that looks at all 
tricky is posix.sgml, which might have to be artificially converted to a 
posix-api.h file just to fit within the Doxygen world.

All of this affects how cygwin-api.pdf and the HTML equivalents get 
built.  My changes so far don't affect these at all.

I assume you've come across Doxygen before in your travels.  I find its 
output far more useful for API references than DocBook.  And, Doxygen is 
in Cygwin, under someone else's maintenance now.

Re: [PATCH] DocBook XML toolchain modernization

Christopher Faylor — 2013-04-30T20:27:37

No, it has to be published but you now have that ability.  The CVS path
is:  :ext:sourceware.org:/cvs/cygwin .


I think the single page form is the only thing required now.


faq.html is a pretty simple file and it seems to work.  Are there any
non-religious advantages to doing this?


This was something that Corinna wanted so I can't comment but since she
asked you to be the maintainer, I'd say that whatever you want is
acceptable as long as we still have a readable FAQ.


We always build everything in Cygwin outside of the source tree so I'd
rather keep autoconf around.  In theory you should be able to just type
"make doc" from wherever you're building cygwin and it should just work.


You have the right to change anything in the doc directory.  Anything
outside of that will require approval.

cgf

Re: [PATCH] DocBook XML toolchain modernization

Warren Young — 2013-04-30T20:09:52

Of course.

Some questions from the original post:

- When a change is checked in to the docs, does it immediately propagate 
to the public web site, or is there a manual publication process?  I 
mean, if the build is technically broken for a few seconds while I check 
in a batch of changes, does it immediately break cygwin.com?  If so, 
that would require that I check in only self-contained change sets even 
if it means long CVS log messages.

- Am I right that we no longer need the second FAQ output format?

- http://cygwin.com/faq.html appears to be assembled from a site-wide 
style/navigation file and winsup/doc/faq/faq.html.  Is that right, and 
if so, do you mind if I add a target to the Makefile that gets you a 
secondary variant of faq.html containing just the <body> tag's contents? 
  Embedding <html> within <html> is eeevil.

- Do you want me to do the proposed doctool to Doxygen conversion, so we 
can get rid of doctool?

- If I get rid of doctool, do you agree that we no longer need Autoconf 
for the docs?

- Is someone using the < at >srcdir< at > feature of the current doc build system 
to build outside the source tree?  (e.g. mkdir foo ; cd foo ; 
../configure && make)  If so, is my "SRCDIR=.. ; make -f ../Makefile" 
alternative acceptable?

(If no one is using the < at >srcdir< at > feature, then we *definitely* don't 
need Autoconf any more after doctool goes away.)

- Any comments about the other items in my FUTURE WORK section? 
Unconditional green light, or do you want to approve them one by one?