gmane.lisp.elephant.devel

Re: (no subject)

Hugo Duncan — 2011-10-13T07:17:56

..Sex all night long!   http://e-aulas.com.ar/com.friend.php?zyrs=18a3

Re: Reason for

Alex Mizrahi — 2011-10-02T16:03:01

 IE> I recall there being problems taking base class injection out of
 IE> shared-initialize - can initialize-instance manipulate the class
 IE> precedence list and direct superclasses at that point in the instance
 IE> initialization?

I think it should work the same way -- INITIALIZE-INSTANCE :around calls 
INITIALIZE-INSTANCE primary method with a modified arglist, which then calls 
SHARED-INITIALIZE with that list.
I don't see why this wouldn't work when SHARED-INITIALIZE works. MOP 
implementation could bypass normal initialization protocol, but then 
SHARED-INITIALIZE method wouldn't work either.

My guess was that it was written this way to avoid duplicating code for 
INITIALIZE-INSTANCE and REINITIALIZE-INSTANCE, if it wasn't meta- stuff 
shared-initialize would be a right place to do it.

 IE> It occurs to me that, the easiest way to fix this is by policy; remove
 IE> the functionality in 'shared-initialize (persistent-metaclass)' from
 IE> the MOP entirely and mandate that persistent classes use th

Re: Reason for

Ian Eslick — 2011-09-29T16:51:06

I recall there being problems taking base class injection out of shared-initialize - can initialize-instance manipulate the class precedence list and direct superclasses at that point in the instance initialization?

It occurs to me that, the easiest way to fix this is by policy; remove the functionality in 'shared-initialize (persistent-metaclass)' from the MOP entirely and mandate that persistent classes use the defpclass macro, or that users inherit from persistent-object manually.  The macro can do the error checking and base-class injection and be done with it.  The tests would have to be updated to stop using the explicit metaclass.

I can make the change and update the tests and then look into the other issues if you concur.

Thanks,
Ian

On Sep 29, 2011, at 12:57 AM, Alex Mizrahi wrote:

Re: Reason for

Alex Mizrahi — 2011-09-29T07:57:37

AMOP spec says that users shouldn't add methods to SHARED-INITIALIZE of 
a metaclass. I think it can be moved to INITIALIZE-INSTANCE  and 
REINITIALIZE-INSTANCE, as they are allowed, and that would also prevent 
problems with internal updates as these functions are not called on 
update-instance-for-*. (I.e. you can get rid of the check of whether 
direct-superclasses were passed or not in that case.)

So that could be a conservative fix (more like bringing it up to the 
spec) which doesn't change the implementation much.

COMPUTE-PRECEDENCE-LIST sounds like a cleaner way to do it (if we don't 
really care for DIRECT superclasses), but I haven't read AMOP on this 
yet, so it is more like a guess.


IIRC both CCL and SBCL were passing all tests but 
CACHING-STYLE-REQUIRED. (Again some MOP part which doesn't work.)

But then I've fixed the test suite itself and got more failures. :)

Can you please check inhibit-rollback (in testconditions) after applying 
my patches? It crashes lisp, as I understand, through

Migration and derived slots

Arnaud Betremieux — 2011-09-28T16:52:21

With the latest Git revision on SBCL 1.0.51, when trying to migrate
objects that have a derived slot, elephant tries to call setf
slot-value-using-class on the derived slot, resulting in an error :
"Cannot write computed (derived) slot... for read/index retrieval
only".

It seems to me that copy-persistent-slots shouldn't try to write
derived slots, as the work will be done when (setf
slot-value-using-class) is called on the slots the derived function
depends on.

So I tried to patch the code for that (see attached patch). This seems
to do the trick in that the migration works.

I stumble upon another problem though : the derived slots are unbound
after migration. Apparently, update-derived-slots is called when
copying slot values, but derived-slots-trigger seems to always return
nil... I don't get why.

Here's my test case if anyone is interested (notice that I need to
require bordeaux-threads manually, which is probably a small
dependency bug ?). The first describe shows c=3, as expected, while
the second

Re: Reason for

Ian Eslick — 2011-09-28T14:40:29

The problem was exposed by loading elephant again without also loading the backend libraries.  I made a few changes so that doesn't happen so easily.  Sounds like you had a better fix.  I haven't dug into this code base, nor done much CL hacking, in almost two years so feeling rusty.  :)

I also made a change for ccl so we don't inject the class if the dependencies argument was not provided (vs passing it along as a nil argument).  I'll look at the alternate placement.

These get ccl to pass all but one test.

Where are you with getting sbcl to build/pass?  I'm testing with 1.0.56

Ian



Sent from my iPhone

On Sep 27, 2011, at 11:57 PM, "Alex Mizrahi" <killerstorm-cDpxXvEWlHUvJsYlp49lxw< at >public.gmane.org> wrote:

Re: Reason for

Alex Mizrahi — 2011-09-28T06:57:40

 IE> Anyone remember why we put this into the elephant.asd file?

 IE> (defmethod operation-done-p ((o load-op) (c elephant-c-source)) nil)

To make sure that library is loaded at least once? There is no way to check 
whether it is loaded via UFFI.
And if you return T ASDF might skip loading altogether.

I've already fixed this, BTW, through tracking what is loaded. Forgot to 
push the patch...

 IE> It was causing the CCL MOP problem discussed earlier.
 IE>   I may have fixes for the CCL MOP - a few tests still fail which I
 IE> will diagnose later.

Have you read openmcl mailing list? It turns out shared-initialize which 
injects persistent-object direct superclass is kinda broken.

BTW, is there a reason to do it via shared-initialize instead of 
compute-precedence-list?

 IE> Part of the win is removing the following line from elephant.asd,
 IE>  but I'm sure I'm casing a regression against another use case this was
 IE> intended to handle.

Hmm, I haven't checked how ASDF handles it by default, I've tri

Reason for

Ian Eslick — 2011-09-28T00:19:39

Anyone remember why we put this into the elephant.asd file?  I think the goal was to force some action to take place so that all the dynamic libraries were loaded each time.  It was causing the CCL MOP problem discussed earlier.  I may have fixes for the CCL MOP - a few tests still fail which I will diagnose later.  

Part of the win is removing the following line from elephant.asd, but I'm sure I'm casing a regression against another use case this was intended to handle.

(defmethod operation-done-p ((o load-op) (c elephant-c-source))
  nil)

Thank you,
Ian

Re: set-valued slots review

Ian Eslick — 2011-09-27T19:25:39

I was creating the set-valued slot API as a placeholder for something more native, but ran out of time to work on the project before really getting it done right.  As you say, the API is awkward.

I think it would be more convenient to use lists as the slot-level API and have a separate API for set operations.

(slot object) -> list | nil
(setf (slot object) list) -> list
(setf (slot object) nil) -> nil
(setf (slot object) atom) -> error or '(atom)

(insert-item 
(remove-item
(find-item 
(map-slot-set 

In general, feel free to use your judgement in upgrading the API - just check with the list that no one is deeply dependent on a particular feature.  Cached slots, pests, set-valued-slots are all beta features that I don't believe are being used seriously.  I'd be open to removing cached slots - they're just too restricting in how they can be used safely in a threaded and/or multi-server environment.

Thanks,
Ian

On Sep 12, 2011, at 8:31 AM, Alex Mizrahi wrote:

Re: Multiple store controllers,with each its own set of classes

Arnaud Betremieux — 2011-09-27T12:13:28

Sorry, I should have been clearer : I *assumed* this bug was the cause,
since there had been an SBCL update in the meantime. Maybe there were
other bugs ? Unfortunately, I don't have the previous version I was
testing on anymore. AFAIK, it was an unmodified, somewhat recent (a few
months) version of Elephant from the darcs repository.

Anyway, my latest test was done with the current git version from
gonzojive on github (had to remove the link to make the message pass the
common-lisp.net spam filter) and everything is OK.

My test was as follows :
- defpclass
- open new BDB store
- make-instance
- get-instances-by-class
- quit
- start a new Lisp instance
- defpclass
- open same BDB store
- get-instances-by-class

With the previous version I was using, get-instances-by-class would return
the correct list the first time, nil the second. Tracing map-index showed
that the btree used was a new one (new oid) at each open-store. With the
current git version, both get-instances-by-class return the correct value.

my

Re: Multiple store controllers,with each its own set of classes

Alex Mizrahi — 2011-09-26T17:05:24

 AB> After looking into this for way too long...It turns out I'm a complete
 AB> idiot : I forgot SBCL had been updated at the same time I changed my
 AB> code to split store controllers, and I was running into the MOP issue
 AB> of new SBCL versions.

Hmm, what SBCL version is that? Can you please check against current git 
version?

I'm aware of problems with class redifinition, but what you've described 
does not look like that.

Re: Multiple store controllers,with each its own set of classes

Arnaud Betremieux — 2011-09-26T12:34:54

After looking into this for way too long...It turns out I'm a complete
idiot : I forgot SBCL had been updated at the same time I changed my code
to split store controllers, and I was running into the MOP issue of new
SBCL versions.

All my apologies.

Multiple store controllers,with each its own set of classes

Arnaud Betremieux — 2011-09-20T07:46:40

Hi all,

I'm trying to use two (BDB) store controllers, each with its own set of
persistent classes. The reason is that I want to be able to modify the
secondary store outside the application (it's read-only within it) and
just plug-in a modified version (close-store, change directory,
open-store) at some point without affecting the rest of the data.

If I just use a let binding to set *store-controller* to the secondary
store around make-instance and get-instances-by-class, everything works,
except I can see the class index is stored in the main store-controller,
which is not good, as I want the secondary store to be independant.

If I set the global *store-controller* to the secondary controller after
having opened both, and even close the first one to make sure before I
call defpclass or any other elephant function, the class index is created
in the right store-controller this time, but I get a new one each time I
open the store (empty btree with a new oid).

I have tried understanding and tracing the rel

status update

Alex Mizrahi — 2011-09-14T14:39:46

Recent patches:

  * set-valued slot tests, optimized postmodern support
  * test-slot-sets enabled in asd

   test-slot-sets thoroughly verifies current behaviour (rather weird)
   optimized postmodern support uses pm-pset for backend storage which 
keeps everything in one btree (database table)

  * disabled testconditions
  * fixes FINISHES macro in tests
  * prevent loading same foreign library (and stuff which "depends" on it)
  * with-interrupts for db-postmodern
  * pm-cursor typo fixed
  * import class-direct-subclasses

Current status: two tests fail both on SBCL 1.0.16 and CCL 1.7:

1. ELEPHANT-DB-PATH-TEST -- as I understand, both tests and function 
ELEPHANT-DB-PATH are buggy as test comments do not quite match 
function's behaviour. I do not quite get how it was supposed to work, so 
I'll leave it as it is for now. I don't quite get why we got this 
failure only now, perhaps reshuffling enabled tests which were not run 
previously (problems from migration to fiveam)

2. CACHING-STYLE-REQUIRED --

bugs in tests

Alex Mizrahi — 2011-09-14T12:30:58

It turns out that for compatibility with RT deftest we redefine 
5am:finishes (not shadowing it) into something which is supposed to work 
with deftest -- it returns NIL if there was error and T otherwise.
There are three problems with it:

1. if `finishes` is in a middle of deftest its output is ignored, i.e. 
it is OK even if there was an ERROR.
2. it just doesn't work in 5am:test, even if you spelled it 5am:finishes
3. And the worst part is that it eats CONDITION as if test was 
successful. And there are conditions which are worse than errors: memory 
corruption, etc.

And this is what happened with inhibit-rollback test -- it crashes CCL 
and makes SBCL signaling memory error.

Besides that, it turns out that cached-test-inherit-ok part of 
caching-style-required did not work, i.e. :cache-style is not inherited.

Re: set-valued slots review

Alex Mizrahi — 2011-09-12T15:31:35

 > IV. Lacking features

Or, alternatively, use lists:

(setf (numbers-of obj) '(1 2 3))
(setf (numbers-of obj) ())

It is kinda asymmetric because accessor will return a slot-set object 
rather than a list, but at least it is more consistent than using setf 
for inserting items.

set-valued slots review

Alex Mizrahi — 2011-09-12T15:26:18

I. RATIONALE

There is a number of cases where one might want to store more than one 
value in a slot. An example from our testassociations: person can hold 
multiple jobs, job can have multiple holders.

There is a number of ways how you can hold these multiple values in a 
single slot:

1. Lisp collection like list or array. It is serialized as a whole, so 
you need to be careful to update slot after manipulations. If there are 
many items and list is frequently updated it is less than perfect.
2. Associations: it is, perhaps, ideal when you have relationship 
between classes. (I'll cover it in another message.)
3. pset (or btree): just assign pset instance to slot, then use pset API 
functions on it.
4. New slot-valued slot feature.

So how what is it? It is very similar to storing pset in a slot, and in 
fact it is implemented as a pset under the hood, with a number of 
differences:

1. There are convenience macros like set-insert and set-remove which 
combine slot-value with pset function. But you can u

Re: CL implementation support

Alex Mizrahi — 2011-09-12T08:58:41

That would be cool, as I'm not comfortable with diving too deep into 
CCL's CLOS implementation.

I've found what happens: if you re-define PERSISTENT-METACLASS (e.g. 
(load "metaclasses.lisp") ) then classes of that metaclass lose their 
slots (CLASS-SLOTS returns NIL) and, perhaps, superclasses too, until 
you redefine them or their parents.

finalize-inheritance doesn't help: it looks like CCL thinks that classes 
are finalized while in fact their content is wiped out.

But what exactly happens with classes when their metaclass is 
redefined... I guess this question is better addressed by guys who wrote 
that CLOS implementation.

...

We trigger this problem because we force re-loading dynamic libraries 
via ASDF:

(defmethod operation-done-p ((o load-op) (c elephant-c-source))
    nil)

So, (asdf:oos 'asdf:load-op :elephant-tests) load Elephant system once.
Then when you work with a certain backend Elephant system it loaded once 
more by that backend's system definition, and ASDF thinks that operation

Re: CL implementation support

Ian Eslick — 2011-09-09T19:14:34

I worked on this project with Clojure, so they may be willing to help diagnose.  I'll be looking into this later this month when I upgrade the running server and add some features to that old project I mentioned.  -Ian

On Sep 9, 2011, at 12:06 PM, Alex Mizrahi wrote:

Re: CL implementation support

Alex Mizrahi — 2011-09-09T19:06:50

The latest available, 1.7, 64-bit. 1.6 had same problems IIRC.

I've localized a problem a bit: it is triggered by recompiling/reloading 
some files in src/elephant. (Maybe it is just one of problems.) So this 
looks like a problem with CCL itself. Or maybe we're breaking something...

I hope that finding which files trigger error will reveal something 
about its nature. For example,

(load (compile-file "src/elephant/classes.lisp"))

is ok. But when classes.fasl is loaded after <something> some slots just 
vanish on class redefinition.

BDB backend seems to be more vulnerable to this stuff, not just slots 
disappear but superclasses... WTF

Re: CL implementation support

Ian Eslick — 2011-09-09T15:53:41

I use CCL for some production CL applications that use Elephant.  Which version of CCL are you having problems with?  I haven't upgraded in awhile - 1.2-1.4 were all good for me.


On Sep 9, 2011, at 3:23 AM, Alex Mizrahi wrote: