gmane.lisp.cclan.general

[PATCH] COMPONENT-DEPENDS-ON and operation subclasses

Richard M Kreuter — 2008-11-07T14:31:09

Hello, The method of COMPONENT-DEPENDS-ON specializing on OPERATION and COMPONENT doesn't do what I'd expect when the operation is an indirect instance of an operation class in a component's IN-ORDER-TO slot. Here's an example involving two SBCL contribs with an inter-system dependency: * (asdf:find-system "sb-md5") # * (asdf:component-depends-on (make-instance 'asdf:load-op) (asdf:find-system "sb-md5")) ((ASDF:COMPILE-OP "sb-md5") (ASDF:LOAD-OP SB-MD5-SYSTEM::SB-ROTATE-BYTE)) * (defclass my-load-op (asdf:load-op) ()) # * (asdf:component-depends-on (make-instance 'my-load-op) (asdf:find-system "sb-md5")) ((ASDF:COMPILE-OP "sb-md5")) The net effect is that subclasses of operation classes don't inherit dependency behavior; in the example above, MY-LOAD-OP isn't able to load sb-md5, because sb-rotate-byte doesn't get loaded. I think it would make sense for operation subclasses to inher

asdf replacement

Faré — 2008-10-18T18:46:59

Note that if you want to develop something that's incompatible with ASDF yet has a migration path from ASDF, you might be interested in hacking XCVB. It's not production-ready yet (and it currently depends on ASDF to be installed itself), but it does its job and brings interesting new features like separate compilation. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Be liberal in what you accept and conservative in what you send. -- Jon Postel ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

a non-contentious patch for ASDF...

Gary King — 2008-10-15T18:54:37

I made a mistake when I applied Richard Kreuter's fix for ASDF's broken circularity detection a while back: I kept in the old ASDF code and added Richard's fix. The patch below removes the code that should have been removed then. With this in, ASDF passes all 14-tests. Index: asdf.lisp =================================================================== RCS file: /cvsroot/cclan/asdf/asdf.lisp,v retrieving revision 1.129 diff -u -w -r1.129 asdf.lisp --- asdf.lisp4 Oct 2008 22:41:04 -00001.129 +++ asdf.lisp15 Oct 2008 18:52:03 -0000 < at >< at > -751,8 +751,6 < at >< at > (if (component-visiting-p operation c) (error 'circular-dependency :components (list c))) (setf (visiting-component operation c) t) - (loop for (required-op . deps) in (component-depends-on operation c) - do (do-dep required-op deps)) (unwind-protect (progn (loop for (required-op . deps) in I've just committed this. -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkw

adding system-loaded-p to ASDF

Gary King — 2008-10-15T19:01:23

AFAIK, there is no _easy_ way to tell if a system has been loaded in ASDF. #'find-system lets me know if a system is defined but I have to look at operation-done-p to know whether or not it has been loaded. The following adds #'system-loaded-p and two methods on #'operation- done-p to make it easier to use by converting symbols into instances of reasonable classes. (in-package #:asdf) (defmethod operation-done-p ((o symbol) (c t)) (operation-done-p (make-instance o) c)) (defmethod operation-done-p ((o t) (c symbol)) (operation-done-p o (find-system c))) (defun system-loaded-p (system) (operation-done-p 'load-op (find-system system))) If there is no disagreement, I'll add some documentation and tests and commit. -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the cool

Re: A plurality of defsystems [was Re: rfc - delaytheparsing of system components ]

GP lisper — 2008-10-14T23:44:30

From: Gary King metabang.com> Date: Tue, 14 Oct 2008 17:59:59 -0400 Lisp tends to make things much harder because we keep inventing things. Each new defsystem adds another barrier to entry because it requires more setup and thinking. Choice is good; too much choice is paralyzing. I must agee with this, having felt this way many times. However, I can also point at UFFI and CFFI as a somewhat similar situation to what is going on in ASDF. The names are similar, the functions are similar, and the latter one improved upon the former. I also know the confusion factor from 'same-name but incompatible with the past'. Pick a rather similar name, and give the ideas the test of time. People do beat pathways to code that moves them in a needed direction. Frankly worrying about the name, rather than the code doesn't seem productive. r ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge

Re: A plurality of defsystems [was Re: rfc - delay theparsing of system components ]

Gary King — 2008-10-14T21:59:59

Hi Richard, How meta! (pun intended) On it's face, I think that your idea is great. I do, however, have two concerns that revolve around your statement that: Sometimes, I think Lisp is nothing but a plethora of ways to do the same thing. Maybe it's only because I'm not familiar with the communities, but AFAIK, if one wants to install something in Ruby, you look for a GEM; if you want to get something for Python, you go to CPAN, etc. Lisp tends to make things much harder because we keep inventing things. Each new defsystem adds another barrier to entry because it requires more setup and thinking. Choice is good; too much choice is paralyzing. To my mind, ASDF has _won_. It is the default and most Lisps are bundled with a version of it. I think that we should be improving it, not replacing it. The last thing we (as a Lisp community) need is another defsystem -- even a 99% compatible one. My concerns therefore are: 1. that this additional layer adds another thing to break and to

A plurality of defsystems [was Re: rfc - delay the parsing of system components ]

Richard M Kreuter — 2008-10-14T14:57:51

For my part, I don't see any reason why the Lisp community should constrain itself to having exactly one defsystem facility. ASDF is useful, but it has shortcomings, and since it's never clear which of ASDF's details are intended or accident, and in any case which details are important to anybody, ISTM that it would probably be a good thing if alternative defsystems could co-exist. So I wonder how hard it would be to introduce an interface that insulated users and programs from the details of how a system's construction and loading happens to be implemented. Something like the following seems as though it would suffice: -- BUILD-SYSTEM &key &allow-other-keys [Function] Call successive functions in *SYSTEM-BUILDER-HOOKS* using , the list of keywords, and :ALLOW-OTHER-KEYS T, until one returns true, and signals an error if no function in *SYSTEM-BUILDER-HOOKS* returns true. *SYSTEM-BUILDER-HOOKS* [Variable] A list of designators for

Re: rfc - delay the parsing of system components

Richard M Kreuter — 2008-10-14T13:08:57

I didn't think my suggestion through. One way to use an ASDF extension is to include package-qualified symbols in the DEFSYSTEM form, and so it's not generally possible to read a DEFSYSTEM form before the relevant extensions are loaded. Example: (defsystem foo :requires "my-asdf-stuff" :components ((:file "foo" :class my-asdf-stuff:whizbang-source-file))) So my suggestion won't work for all cases. Sorry about that. -- Richard ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Robert Goldman — 2008-10-14T01:14:06

Nitpicky suggestion --- let's call it something else to avoid confusing with the (deprecated) CL notion of require. While we're messing with this, any chance of getting something which wraps around the whole operation on a component so that we can, e.g., impose dynamic bindings around the entire load-op as applied to a system? I suppose that's off-topic, but the current overhaul seems like a good opportunity to get that done as well.... This seems like the preferable solution. ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Gary King — 2008-10-14T00:06:49

OK. I hear what everyone is saying and appreciate the feedback. I'd like to back up (or out!) a bit and reconsider my goals: * I don't want to replace ASDF; it;s good, it works, it's in use; adding a new one just muddies the waters. * I do want to make it easier to use extensions (new operations and file types) in a system file. My first plan was the proposal to delay parsing and muck with traverse / perform. Here is two others: 1. add keyword * As in #0, Delay the parsing of system components (so that these can come later) * Add a new keyword (as RIchard Kreuter suggested) that loads extensions and which is processed "outside" of and before traverse/ perform. Let's call it :requires * An operation will first process the requires form, then expand the component forms, then run the existing traverse/perform 2. Like #1 only without the new keyword. * ASDF behaves as it does now except that a systems :depends-on is processed before and separately from the processing of its components. As

Re: rfc - delay the parsing of system components

Daniel Barlow — 2008-10-13T20:27:41

Excuse me if I'm about to rehash old ground. I cannot entirely remember the reasons for plan-then-execute, but asdf was changed to work that way very early in its history and there must have been some kind of justification at the time. One point is that if you want to write operation-done-p methods which compare file dates then it's much easier to think about if you're comparing the dates from *before* you started work than from some arbitrary point halfway through the operation - it's also safer in the face of clock drift. KMP does the the same thing in his "Description of Large Systems" paper, though doesn't give much justification for it beyond "Because a system can be asked to produce a plan for an operation such as compilation without actually performing the operation, it is possible to write programs which inspect the plan, possibly optimizing it or presenting it for for interactive approval, before executing it. " http://www.nhplace.com/kent/Papers/Large-Systems.html A lot of this was driven by

Re: rfc - delay the parsing of system components

Nikodemus Siivola — 2008-10-13T19:16:51

I agree. "plan, then execute" is one of the major (mis?)features of ASDF. While I have often wanted it, calling a system that behaves like that ASDF is just confusing. While the change may be practically backwards-incompatible, it is a major headache to deal with when people start actually using it -- since system depending on it will *not* work on older ASDF versions. YASDF, ASDF2, ASDF.REC, GKSDF, whatever -- and I *like* the idea, but not calling it ASDF. Cheers, -- Nikodemus ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Christophe Rhodes — 2008-10-13T13:08:14

Yes, I do, because (as I understand it) it means that performing operations on components now wraps operations on subcomponents, rather than happening afterwards. If this causes no difficulties in the wild, then it also costs you very little to call this "GHJK", provide an in-the-wild-compatible "ASDF" package, and advertise 99.9% (or whatever) ASDF-compatibility, without the horrible broken bits and with additional capabilities. The cost of not doing so is, in my experience, huge: over time, references to "ASDF"-the-software accumulate, and suddenly no-one knows whether a particular reference applies to before or after your change, or both. I don't see why you even _want_ to keep the same name, frankly. Best, Christophe ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event a

Re: rfc - delay the parsing of system components

Gary King — 2008-10-13T12:51:27

The question reduces to whether or not this is a different species. I don't think it is. Richard correctly points out that my proposal might cause problems in the "wild" but there is no evidence for this (yet...). To my mind, I've kept the same syntax and ASDF does the same operations in the same order (at least in as much as ASDF ever did the same things in the same order...). The only change is that instead of planning everything before executing, ASDF would now execute as it went. I don't see this as incompatible. Do you? thanks, -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_i

Re: rfc - delay the parsing of system components

Christophe Rhodes — 2008-10-13T06:33:42

No problem with that, but biological taxonomists have learnt that it's useful to give mutually-incompatible species different names. Some day, computer programmers might see the point of that, too. (In other words: please do make code easier to write, but please also give the resulting software a different name.) Cheers, Christophe ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Gary King — 2008-10-13T00:08:04

Hi Richard, et. al. Thanks for your comments and thoughts. This is true but I have my doubts that it will matter. Methods like COMPONENT-DEPENDS-ON shouldn't really have anything to do with the environment (either X depends on Y or it doesn't). Methods like operation-done-p should depend on a component and its children (not its parents) and thus would not be altered by the proposed change. It's true that my proposal isn't required to make ASDF jump through the hoops I want it to. IMHO, the eval-when solution is ugly and while adding another slot is alright, it seems to me that I should be able to use operations and components and such defined by a system's dependencies. I.e., that what I'm proposing is the way things should be. Obviously, reasonable people can disagree! Hmmm, evolve or die . There is always more code to write in the future than has been written in the past. This proposed change makes ASDF simpler in a few ways and, arguably, a bit easier to understand. I

Re: rfc - delay the parsing of system components

Richard M Kreuter — 2008-10-08T15:12:04

In the plan-before-execute design that ASDF's always had, methods that implement pieces of the plan construction protocol (OPERATION-DONE-P, COMPONENT-DEPENDS-ON, et al.) can rely on being executed before any steps are carried out in the current run. Since the protocol doesn't have any solid semantics, extensions' methods can do anything, really; executing them mid-way through execution can expose them to conditions that are arbitrarily unlike what such methods have been written to reason about. The idiomatic solution for this is to load ASDF extensions in an EVAL-WHEN at the beginning of a .asd file. But maybe what you're after is representing the fact that system FOO needs ASDF extension BAR, which isn't explicit with the EVAL-WHEN solution. If that's what you need, it might suffice to add a slot for this to SYSTEM and to modify PARSE-COMPONENT-FORM to hoist out the slot's initarg and do the LOAD-OP on those extensions before looping over the components. I think #2 is an at-least-in-theory incompat

rfc - delay the parsing of system components

Gary King — 2008-10-08T14:12:00

I just finished a rough cut of an ASDF overall that makes three largish changes: 1. delays the parsing of system components until needed by traverse. In particular, this means that the :components clause of defsystem form happens after that system's dependencies have been loaded rather than when the ASD file itself is loaded. 2. rewrites traverse as a set of methods that immediately call perform (rather than building up a list of perform actions that are then called in operate). 3. does away with :do-first and moves what it did into :in-order-to The main motivation for change #1 is to be able to define new operations and source file classes in a system's dependencies and have them used in that systems definitions; the main motivation#2 was #1: it does no good to delay parsing of the defsystem form if traverse has to complete the parse before anything else happens!. As for #3, while doing three things at once is just very human (and besides, it made #2 easier). I've got several h

Re: in-order-to and/or do-first

Andreas Fuchs — 2008-10-08T07:50:14

The separation between the two as it is right now is used by what I'll call compilation dependencies: Compilation of file A will trigger compilation of dependent file B if load-op is performed on the system containing them. I tried to pry in-order-to and do-first apart once, then just gave up and left things as they were, because I could not figure out how to guarantee that compilation of dependent files happens before loading in this case. HTH,

Re: in-order-to and/or do-first

Richard M Kreuter — 2008-10-08T01:22:18

This came up a little more than a year ago: http://article.gmane.org/gmane.lisp.cclan.general/674 -- Richard ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

in-order-to and/or do-first

Gary King — 2008-10-07T23:45:55

Can anyone recall why we have both in-order-to and do-first? Doesn't in-order-to subsume do-first? thanks, -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/