gmane.lisp.cclan.general

[PATCH] COMPONENT-DEPENDS-ON and operation subclasses

Richard M Kreuter — 2008-11-07T14:31:09

Hello, The method of COMPONENT-DEPENDS-ON specializing on OPERATION and COMPONENT doesn't do what I'd expect when the operation is an indirect instance of an operation class in a component's IN-ORDER-TO slot. Here's an example involving two SBCL contribs with an inter-system dependency: * (asdf:find-system "sb-md5") # * (asdf:component-depends-on (make-instance 'asdf:load-op) (asdf:find-system "sb-md5")) ((ASDF:COMPILE-OP "sb-md5") (ASDF:LOAD-OP SB-MD5-SYSTEM::SB-ROTATE-BYTE)) * (defclass my-load-op (asdf:load-op) ()) # * (asdf:component-depends-on (make-instance 'my-load-op) (asdf:find-system "sb-md5")) ((ASDF:COMPILE-OP "sb-md5")) The net effect is that subclasses of operation classes don't inherit dependency behavior; in the example above, MY-LOAD-OP isn't able to load sb-md5, because sb-rotate-byte doesn't get loaded. I think it would make sense for operation subclasses to inherit dependency behavior (patch attached), but maybe there's some reason why they don't. Can anybody think up a story under which the current implementation is the right thing? Thanks, Richard --- asdf.lisp15 Oct 2008 18:56:08 -00001.130 +++ asdf.lisp7 Nov 2008 14:22:46 -0000 < at >< at > -640,8 +640,7 < at >< at > (component-depends-on (make-instance op-spec) c)) (defmethod component-depends-on ((o operation) (c component)) - (cdr (assoc (class-name (class-of o)) - (slot-value c 'in-order-to)))) + (cdr (assoc o (slot-value c 'in-order-to) :test #'typep))) (defgeneric component-self-dependencies (operation component)) ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________ cclan-list mailing list cclan-list< at >lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/cclan-list

asdf replacement

Faré — 2008-10-18T18:46:59

Note that if you want to develop something that's incompatible with ASDF yet has a migration path from ASDF, you might be interested in hacking XCVB. It's not production-ready yet (and it currently depends on ASDF to be installed itself), but it does its job and brings interesting new features like separate compilation. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Be liberal in what you accept and conservative in what you send. -- Jon Postel ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

a non-contentious patch for ASDF...

Gary King — 2008-10-15T18:54:37

I made a mistake when I applied Richard Kreuter's fix for ASDF's broken circularity detection a while back: I kept in the old ASDF code and added Richard's fix. The patch below removes the code that should have been removed then. With this in, ASDF passes all 14-tests. Index: asdf.lisp =================================================================== RCS file: /cvsroot/cclan/asdf/asdf.lisp,v retrieving revision 1.129 diff -u -w -r1.129 asdf.lisp --- asdf.lisp4 Oct 2008 22:41:04 -00001.129 +++ asdf.lisp15 Oct 2008 18:52:03 -0000 < at >< at > -751,8 +751,6 < at >< at > (if (component-visiting-p operation c) (error 'circular-dependency :components (list c))) (setf (visiting-component operation c) t) - (loop for (required-op . deps) in (component-depends-on operation c) - do (do-dep required-op deps)) (unwind-protect (progn (loop for (required-op . deps) in I've just committed this. -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

adding system-loaded-p to ASDF

Gary King — 2008-10-15T19:01:23

AFAIK, there is no _easy_ way to tell if a system has been loaded in ASDF. #'find-system lets me know if a system is defined but I have to look at operation-done-p to know whether or not it has been loaded. The following adds #'system-loaded-p and two methods on #'operation- done-p to make it easier to use by converting symbols into instances of reasonable classes. (in-package #:asdf) (defmethod operation-done-p ((o symbol) (c t)) (operation-done-p (make-instance o) c)) (defmethod operation-done-p ((o t) (c symbol)) (operation-done-p o (find-system c))) (defun system-loaded-p (system) (operation-done-p 'load-op (find-system system))) If there is no disagreement, I'll add some documentation and tests and commit. -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: A plurality of defsystems [was Re: rfc - delaytheparsing of system components ]

GP lisper — 2008-10-14T23:44:30

From: Gary King metabang.com> Date: Tue, 14 Oct 2008 17:59:59 -0400 Lisp tends to make things much harder because we keep inventing things. Each new defsystem adds another barrier to entry because it requires more setup and thinking. Choice is good; too much choice is paralyzing. I must agee with this, having felt this way many times. However, I can also point at UFFI and CFFI as a somewhat similar situation to what is going on in ASDF. The names are similar, the functions are similar, and the latter one improved upon the former. I also know the confusion factor from 'same-name but incompatible with the past'. Pick a rather similar name, and give the ideas the test of time. People do beat pathways to code that moves them in a needed direction. Frankly worrying about the name, rather than the code doesn't seem productive. r ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: A plurality of defsystems [was Re: rfc - delay theparsing of system components ]

Gary King — 2008-10-14T21:59:59

Hi Richard, How meta! (pun intended) On it's face, I think that your idea is great. I do, however, have two concerns that revolve around your statement that: Sometimes, I think Lisp is nothing but a plethora of ways to do the same thing. Maybe it's only because I'm not familiar with the communities, but AFAIK, if one wants to install something in Ruby, you look for a GEM; if you want to get something for Python, you go to CPAN, etc. Lisp tends to make things much harder because we keep inventing things. Each new defsystem adds another barrier to entry because it requires more setup and thinking. Choice is good; too much choice is paralyzing. To my mind, ASDF has _won_. It is the default and most Lisps are bundled with a version of it. I think that we should be improving it, not replacing it. The last thing we (as a Lisp community) need is another defsystem -- even a 99% compatible one. My concerns therefore are: 1. that this additional layer adds another thing to break and to confuse people. 2. that building a new defsystem adds another thing for people to do if they want to use system _foo_ that uses it. That said, I agree that incompatible changes should not be introduced without a lot of thought (that's why noone commits to ASDF unless there is a discussion here first, right?). -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

A plurality of defsystems [was Re: rfc - delay the parsing of system components ]

Richard M Kreuter — 2008-10-14T14:57:51

For my part, I don't see any reason why the Lisp community should constrain itself to having exactly one defsystem facility. ASDF is useful, but it has shortcomings, and since it's never clear which of ASDF's details are intended or accident, and in any case which details are important to anybody, ISTM that it would probably be a good thing if alternative defsystems could co-exist. So I wonder how hard it would be to introduce an interface that insulated users and programs from the details of how a system's construction and loading happens to be implemented. Something like the following seems as though it would suffice: -- BUILD-SYSTEM &key &allow-other-keys [Function] Call successive functions in *SYSTEM-BUILDER-HOOKS* using , the list of keywords, and :ALLOW-OTHER-KEYS T, until one returns true, and signals an error if no function in *SYSTEM-BUILDER-HOOKS* returns true. *SYSTEM-BUILDER-HOOKS* [Variable] A list of designators for functions that take a string designator and a list of keywords. When called, each function will attempt to build a system identified by the string designator and possibly some of the keyword arguments, and return true if such a system been built, or NIL otherwise. Building a system might have arbitrary side-effects on the Lisp instance, but may not cause it to terminate. LOAD-SYSTEM &key &allow-other-keys [Function] Call successive functions in *SYSTEM-LOADER-HOOKS* using , the list of keywords, and :ALLOW-OTHER-KEYS T until one returns true, and signals an error if no function in *SYSTEM-LOADER-HOOKS* returns true. *SYSTEM-LOADER-HOOKS* [Function] A list of designators for functions that take a string designator and a list of keywords. When called, each function will attempt to load a system identified by the string designator and possibly some of the keyword arguments. -- So, for example, ASDF could install a hook for each of the two variables that called OPERATE with suitable arguments, and other defsystems could do analogous things. Library management tools like asdf-install could indirect through BUILD-SYSTEM and LOAD-SYSTEM. Inter-system dependencies could translate into to calls to LOAD-SYSTEM (possibly after trying a defsystem-specific mechanism like what ASDF does now). Since both of these functions takes keywords, things like system versions or arguments to the construction or loading machinery can be supplied and their interpretation is left to an underlying defsystem tool. Thoughts? -- Richard ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Richard M Kreuter — 2008-10-14T13:08:57

I didn't think my suggestion through. One way to use an ASDF extension is to include package-qualified symbols in the DEFSYSTEM form, and so it's not generally possible to read a DEFSYSTEM form before the relevant extensions are loaded. Example: (defsystem foo :requires "my-asdf-stuff" :components ((:file "foo" :class my-asdf-stuff:whizbang-source-file))) So my suggestion won't work for all cases. Sorry about that. -- Richard ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Robert Goldman — 2008-10-14T01:14:06

Nitpicky suggestion --- let's call it something else to avoid confusing with the (deprecated) CL notion of require. While we're messing with this, any chance of getting something which wraps around the whole operation on a component so that we can, e.g., impose dynamic bindings around the entire load-op as applied to a system? I suppose that's off-topic, but the current overhaul seems like a good opportunity to get that done as well.... This seems like the preferable solution. ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Gary King — 2008-10-14T00:06:49

OK. I hear what everyone is saying and appreciate the feedback. I'd like to back up (or out!) a bit and reconsider my goals: * I don't want to replace ASDF; it;s good, it works, it's in use; adding a new one just muddies the waters. * I do want to make it easier to use extensions (new operations and file types) in a system file. My first plan was the proposal to delay parsing and muck with traverse / perform. Here is two others: 1. add keyword * As in #0, Delay the parsing of system components (so that these can come later) * Add a new keyword (as RIchard Kreuter suggested) that loads extensions and which is processed "outside" of and before traverse/ perform. Let's call it :requires * An operation will first process the requires form, then expand the component forms, then run the existing traverse/perform 2. Like #1 only without the new keyword. * ASDF behaves as it does now except that a systems :depends-on is processed before and separately from the processing of its components. As I see it, #2 this keeps the syntax the same but runs a higher risk of breaking things whereas #1 adds syntax but can cause no breakage. Both of these lose the ability to (easily) wrap restarts around modules or systems. -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Daniel Barlow — 2008-10-13T20:27:41

Excuse me if I'm about to rehash old ground. I cannot entirely remember the reasons for plan-then-execute, but asdf was changed to work that way very early in its history and there must have been some kind of justification at the time. One point is that if you want to write operation-done-p methods which compare file dates then it's much easier to think about if you're comparing the dates from *before* you started work than from some arbitrary point halfway through the operation - it's also safer in the face of clock drift. KMP does the the same thing in his "Description of Large Systems" paper, though doesn't give much justification for it beyond "Because a system can be asked to produce a plan for an operation such as compilation without actually performing the operation, it is possible to write programs which inspect the plan, possibly optimizing it or presenting it for for interactive approval, before executing it. " http://www.nhplace.com/kent/Papers/Large-Systems.html A lot of this was driven by IRC discussion with Kevin Rosenberg - I wish I'd kept notes in some more easily-searchable format :-( I don't have a strong opinion these days on which approach is "better", but I do agree that the change will break code in the wild, and should imply a name change if made. -dan ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Nikodemus Siivola — 2008-10-13T19:16:51

I agree. "plan, then execute" is one of the major (mis?)features of ASDF. While I have often wanted it, calling a system that behaves like that ASDF is just confusing. While the change may be practically backwards-incompatible, it is a major headache to deal with when people start actually using it -- since system depending on it will *not* work on older ASDF versions. YASDF, ASDF2, ASDF.REC, GKSDF, whatever -- and I *like* the idea, but not calling it ASDF. Cheers, -- Nikodemus ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Christophe Rhodes — 2008-10-13T13:08:14

Yes, I do, because (as I understand it) it means that performing operations on components now wraps operations on subcomponents, rather than happening afterwards. If this causes no difficulties in the wild, then it also costs you very little to call this "GHJK", provide an in-the-wild-compatible "ASDF" package, and advertise 99.9% (or whatever) ASDF-compatibility, without the horrible broken bits and with additional capabilities. The cost of not doing so is, in my experience, huge: over time, references to "ASDF"-the-software accumulate, and suddenly no-one knows whether a particular reference applies to before or after your change, or both. I don't see why you even _want_ to keep the same name, frankly. Best, Christophe ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Gary King — 2008-10-13T12:51:27

The question reduces to whether or not this is a different species. I don't think it is. Richard correctly points out that my proposal might cause problems in the "wild" but there is no evidence for this (yet...). To my mind, I've kept the same syntax and ASDF does the same operations in the same order (at least in as much as ASDF ever did the same things in the same order...). The only change is that instead of planning everything before executing, ASDF would now execute as it went. I don't see this as incompatible. Do you? thanks, -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Christophe Rhodes — 2008-10-13T06:33:42

No problem with that, but biological taxonomists have learnt that it's useful to give mutually-incompatible species different names. Some day, computer programmers might see the point of that, too. (In other words: please do make code easier to write, but please also give the resulting software a different name.) Cheers, Christophe ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Gary King — 2008-10-13T00:08:04

Hi Richard, et. al. Thanks for your comments and thoughts. This is true but I have my doubts that it will matter. Methods like COMPONENT-DEPENDS-ON shouldn't really have anything to do with the environment (either X depends on Y or it doesn't). Methods like operation-done-p should depend on a component and its children (not its parents) and thus would not be altered by the proposed change. It's true that my proposal isn't required to make ASDF jump through the hoops I want it to. IMHO, the eval-when solution is ugly and while adding another slot is alright, it seems to me that I should be able to use operations and components and such defined by a system's dependencies. I.e., that what I'm proposing is the way things should be. Obviously, reasonable people can disagree! Hmmm, evolve or die . There is always more code to write in the future than has been written in the past. This proposed change makes ASDF simpler in a few ways and, arguably, a bit easier to understand. It also allows for restarts like "recompile module 'foo'" or "recompile system 'bar'" that the plan-before-execute design makes very difficult (because the plan more or less obliterates the dependency structure). There is still more cleanup to do but I think I'll post the source somewhere that people can download and try and see if it breaks anything. So far, the changed version is liking all the systems I've tossed its way. thanks again, -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: rfc - delay the parsing of system components

Richard M Kreuter — 2008-10-08T15:12:04

In the plan-before-execute design that ASDF's always had, methods that implement pieces of the plan construction protocol (OPERATION-DONE-P, COMPONENT-DEPENDS-ON, et al.) can rely on being executed before any steps are carried out in the current run. Since the protocol doesn't have any solid semantics, extensions' methods can do anything, really; executing them mid-way through execution can expose them to conditions that are arbitrarily unlike what such methods have been written to reason about. The idiomatic solution for this is to load ASDF extensions in an EVAL-WHEN at the beginning of a .asd file. But maybe what you're after is representing the fact that system FOO needs ASDF extension BAR, which isn't explicit with the EVAL-WHEN solution. If that's what you need, it might suffice to add a slot for this to SYSTEM and to modify PARSE-COMPONENT-FORM to hoist out the slot's initarg and do the LOAD-OP on those extensions before looping over the components. I think #2 is an at-least-in-theory incompatible change. It might not break anything in the wild, but that can't be known in advance. It might also lead to a better defsystem than ASDF, but I'm not sure that ASDF should be breaking compatibility with itself at this point. -- Richard ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

rfc - delay the parsing of system components

Gary King — 2008-10-08T14:12:00

I just finished a rough cut of an ASDF overall that makes three largish changes: 1. delays the parsing of system components until needed by traverse. In particular, this means that the :components clause of defsystem form happens after that system's dependencies have been loaded rather than when the ASD file itself is loaded. 2. rewrites traverse as a set of methods that immediately call perform (rather than building up a list of perform actions that are then called in operate). 3. does away with :do-first and moves what it did into :in-order-to The main motivation for change #1 is to be able to define new operations and source file classes in a system's dependencies and have them used in that systems definitions; the main motivation#2 was #1: it does no good to delay parsing of the defsystem form if traverse has to complete the parse before anything else happens!. As for #3, while doing three things at once is just very human (and besides, it made #2 easier). I've got several hours of work left cleaning up loose ends and moving restarts and with-compilation-units around into better places. What I'd appreciate is any high-level feedback about the idea of these changes and whether or not I've missed some horrible interaction that nullifies the whole direction. I hope to post the suggested changes later in the week. Until then, happy coding. -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Re: in-order-to and/or do-first

Andreas Fuchs — 2008-10-08T07:50:14

The separation between the two as it is right now is used by what I'll call compilation dependencies: Compilation of file A will trigger compilation of dependent file B if load-op is performed on the system containing them. I tried to pry in-order-to and do-first apart once, then just gave up and left things as they were, because I could not figure out how to guarantee that compilation of dependent files happens before loading in this case. HTH,

Re: in-order-to and/or do-first

Richard M Kreuter — 2008-10-08T01:22:18

This came up a little more than a year ago: http://article.gmane.org/gmane.lisp.cclan.general/674 -- Richard ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

in-order-to and/or do-first

Gary King — 2008-10-07T23:45:55

Can anyone recall why we have both in-order-to and do-first? Doesn't in-order-to subsume do-first? thanks, -- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/