gmane.lisp.bordeaux-threads.devel

Re: Deprecating recursive locks

Antony — 2012-05-03T07:07:01

I had very similar thought. I think in general the problem is lots of 
people try to write up software engineering rules based on the end 
product (that is the code as it looks while being shipped). Whereas the 
rules/guidelines should be done based on the process that gets you from 
nothing to the end product. This is true no matter how rapid (or not)  
the methodology is.

-Antony

Re: Deprecating recursive locks

Antony — 2012-05-03T07:00:37

While what you say below I am sure is true, is it good enough to say 
recursive locking should be removed?
Trying to use the same mutex for normal locking and again as part of 
condition variable structure I think is breaking the abstraction of 
condition variable.
Also, if someone writes code that recursively locks to control access to 
a resource and then tries to do signalling deep in the call stack using 
that same mutex combined with a condition var, that's just bad code. 
Saying we shouldn't have recursive mutexes because of that is like 
saying we shouldn't have threads cause they can corrupt each others 
variables.

I do like this mail thread since I think it's good to discuss this. 
Sorry, I did not respond early enough.

-Antony

Re: Deprecating recursive locks

Leslie P. Polzer — 2012-05-03T06:28:30

I think it's a bad idea. For me rapid prototyping is an important
part of Lisp, and recursive locks support this approach. What
Butenhof basically holds against them is that you're not looking
at your threading code closely enough. But in the beginning stages
of a project I actually might not want to do that. Doing away
with them would be premature optimization. Add a note of caution
to the docstring if you're that concerned, but don't remove them.

  Leslie

Re: Deprecating recursive locks

James M. Lawrence — 2012-05-03T04:28:32

Ensuring portability seems outside the scope of bordeaux. Take the
example of Clozure, where all locks are recursive locks. Is it
bordeaux's responsibility to ensure that the thing returned by
make-lock is not locked twice?

Indeed bordeaux includes functions which are explicitly unportable.
The issue you mention with condition variables is a good reason to
update the documentation on recursive locks. Why not simply note the
behavior as unportable, as with destroy-thread?

Nobody is arguing to remove the five functions which "are not advised
to be called from normal user code". They are kept because they are
useful. The same is true of recursive locks, which for example are
convenient for logging. Isn't that reason enough to keep them?

It appears that the 11 implementations covered by bordeaux all support
recursive locks except Macintosh CL. All locking is recursive in
Clozure and CMUCL. Allegro has

  (mp:with-process-lock (lock :norecursive nil) ...).

I've had no trouble with SBCL's non-POSIX recursive

Re: Deprecating recursive locks

Jean-Claude Beaudoin — 2012-05-03T02:30:25

I had not paid much attention to this issue until now but this
last argument about condition variables just made such that
the default behavior of #'mt:make-lock of MKCL will be from now on
to produce a non-recursive lock even if this is a break with
the behavior inherited from ECL. This starting with MKCL 1.1.0 RC2.
If you want recursive locks you will have to request it
explicitly with (make-lock :recursive t).

There may be some protest against this change but the strength
of the "condition variable" point will override them.

Cheers,

Jean-Claude Beaudoin

P.S.: SLIME blew up big time on this change. I had to force
swank-backend:make-lock back into producing recursive locks
in order to prevent it from crashing. That's bad...

I am also considering whether #'mt:condition-wait
should emit a warning if it ever receives a recursive lock.
(Even more so if the recursive lock has a count above 1!)
What do you think of this?

Re: Deprecating recursive locks

Vladimir Sedach — 2012-05-02T20:57:44

Another argument against recursive locks that hasn't come up yet is
that they fail very badly when used with condition variables. I think
the fact that recursive locks aren't supported by most implementations
plus the fact that they behave differently wrt condition variables on
different implementations and operating systems are very good
arguments for leaving recursive locks out of a portable threading
library.

Vladimir

On Wed, May 2, 2012 at 3:10 PM, Antony <lisp.linux-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

Re: Deprecating recursive locks

Antony — 2012-05-02T19:10:39

I use CCL at the moment
Not having recursive locks makes locking code more complicated. With 
recursive locks I only need to think about ensuring order when more than 
one lock is involved.  Without recursive lock, every possible call path 
to every lock needs to be thought through.

I confirmed that CCL does support recursive locking
http://clozure.com/pipermail/openmcl-devel/2012-May/013536.html

FWIW  for .net
   http://msdn.microsoft.com/en-us/library/de0542zz%28v=vs.71%29.aspx
   "It is legal for the same thread to invoke Enter more than once 
without it blocking"

   for java  
http://docs.oracle.com/javase/tutorial/essential/concurrency/locksync.html
   "a thread can acquire a lock that it already owns"  (so I assume ABCL 
does recursive locks)

I read the message by Dave Butenhof
  http://groups.google.com/group/comp.programming.threads/msg/d835f2f6ef8aed99?hl=en&pli=1
that I think was pointed as the logic behind getting rid of recursive 
mutexes in bordeaux.
I did a basic read. I am not an expert. I

Re: Deprecating recursive locks

Stelian Ionescu — 2012-05-02T19:09:23

Indeed I was fooled by that. Good to know, this is a excellent reason
not to ever use Clozure

Re: Deprecating recursive locks

James M. Lawrence — 2012-05-02T17:13:57

log4cl, released a few months ago.

You mentioned that Clozure does not support recursive locks. The funny
thing about Clozure is that ccl:make-lock actually returns a recursive
lock. So bt:make-recursive-lock is, through coincidence, providing a
recursive lock by calling ccl:make-lock.

Thus users may be fooled into thinking bordeaux-threads is making some
guarantee when it hasn't. The behavior of Clozure may change in the
future, for all we know.

The best of both worlds is to move recursive locks to the
introspection/debugging section of the documentation. Recursive locks
would be there as convenience for the use case I mentioned, while
users would be made aware that they are not dependable across
implementations.

Re: Deprecating recursive locks

Stelian Ionescu — 2012-05-02T15:27:51

Which one ? Such library is broken because few CL implementations
actually have recursive locks

Re: Deprecating recursive locks

James M. Lawrence — 2012-05-02T15:15:21

I currently use a recursive lock for debug logging. In the past I
wrapped *debugger-hook* with a recursive lock in order to avoid
multiple simultaneous debugger popups. Both are cases of throwaway
code.

David Butenhof does mention that recursive locks "can be a great tool"
as long as they are properly understood as a temporary measure. That's
how I use them, and others may as well. Yes, they are a hack, but they
are a convenient hack.

If recursive locks are removed then I would expect people to write
their own half-ass implementations, as I would do for my debug logger.
I agree that using them in a non-temporary context may constitute poor
design, however that does not imply that they should be removed.

Perhaps adding a note to the documentation regarding the hackiness of
recursive locks (and/or even linking to the Butenhof post) would
suffice in lieu of outright deletion?

If you do decide to remove them, consider issuing deprecation warnings
in the next release (or longer) before the actual removal. Th

Re: Deprecating recursive locks

Stelian Ionescu — 2012-04-22T14:41:03

That post contains a good analysis of the problem, but then gives the
worst possible advice to solve it



Point 5 says "Major version zero (0.y.z) is for initial development.
Anything may change at any time. The public API should not be considered
stable". So no problem there

Re: [patch] LispWorks 6.x MP updates + EOS

Stelian Ionescu — 2012-04-21T21:41:54

I now maintain FiveAM(http://fenlix.dreamwidth.org/2255.html) and I
removed the dependency on Arnesi, so you can expect to see it in the
next release of Quicklisp.

Re: CONDITION-WAIT with timeout

Thomas Munro — 2012-04-19T07:28:12

Ok -- so I guess I need to write a patch that covers as many
Lisps as I can get my hands on for testing, and a test that
demonstrates that it works correctly.

Before I go ahead and do that, below is an example showing what
the ABCL proposed implementation would look like, for feedback.
Some questions and notes:

1.  I'm using an &optional argument which seems the tidiest to
me.  Would anyone prefer to have a separate function instead, say
CONDITION-TIMED-WAIT, or a &key?

2.  The ABCL implementation cannot tell you whether the wait
returned because of timeout (although others can), so to help people
write portable code the return value would have to remain
undefined.

3.  For those implementations which can't support it yet, it
might be an option to simply not accept the &optional argument,
so that programs that need it fail to compile rather than failing
at runtime on implementations that lack it.  Alternatively I
could make those implementation raise an error if TIMEOUT is not
NIL.  I guess timeout suppo

Re: CONDITION-WAIT with timeout

Stelian Ionescu — 2012-04-18T09:56:42

Then I'd be grateful if you could add timeout support and send us a
patch :)

Re: Deprecating recursive locks

Attila Lendvai — 2012-04-18T04:15:14


IMHO it's not a good practice and it does cost complexity/noise.

you lose the clarity around the digital representation(s) of the
identity of the library. there's only one b-t library, and the fact
that it has changed in a way that made it incompatible with something
else does not change its identity.

but it's only the 0.02 of an outsider...

Re: CONDITION-WAIT with timeout

Thomas Munro — 2012-04-18T00:20:19

I went through the existing implementations and googled for
timeout variants of the mechanisms being used and found these:

ABCL: (threads:object-wait condition timeout)
Allegro: ?
Clisp: (mt:exemption exemption mutex :timeout timeout)
Clozure: (ccl:timed-wait-on-semaphore semaphore timeout)
CMUCL: ?
Corman: ?
ECL: (mp:condition-variable-timedwait condition mutex timeout)
LispWorks simulated: ?
LispWorks 6: (mp:condition-variable-wait condition mutex :timeout timeout)
MCL: ?
SBCL: (sb-thread:condition-wait condition mutex :timeout timeout)
SCL: (thread:cond-var-timedwait condition mutex timeout)

I couldn't immediately grok those with question marks, but I
suspect it's possible for most of them though.  I hope someone
who knows those systems can comment.

I don't have any suggestion on what to do if there is no
implementation apart from the obvious and unsatisfactory 'raise
an error at runtime'.

Re: Deprecating recursive locks

Anton Vodonosov — 2012-04-17T21:48:33


17.04.2012, 20:14, "Vladimir Sedach" <vsedach-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>:

If the public API is changed in a not-backward-compatible way, I would 
suggest to release new ASDF system (bordeaux-threads2), and
leave the old version available in quicklisp forever.

People who are ready to use new version, just add bordeaux-thread2 
into their ASDF dependency. Others are able to use old version.

This is a good practice. And it costs nothing.

Some links on this subject:
http://lispcaveats.tumblr.com/post/13259176455/ffi-linking-against-shared-libraries
http://semver.org/

Re: Deprecating recursive locks

Anton Vodonosov — 2012-04-17T19:09:00


17.04.2012, 19:21, "Stelian Ionescu" <sionescu-ct8TMaJCiNI< at >public.gmane.org>:

Does it mean that every time, when trying to lock an object, 
we will need to ensure it hasn't been locked already?

If yes, it complicates writing synchronization code.

But if it can not be reliably supported based on the features provided
by all lisps, then removing them may probably be right.

Re: Deprecating recursive locks

Stelian Ionescu — 2012-04-17T16:17:30

A month or two, perhaps. It depends on my spare time

Re: Deprecating recursive locks

Vladimir Sedach — 2012-04-17T16:14:31

I think it is a very good idea.

When are you planning the 1.0 release? It would be nice to go through
all the systems dependent on bordeaux-threads in Quicklisp and notify
the maintainers of any that use recursive locks to fix their code.

Vladimir

On Tue, Apr 17, 2012 at 11:21 AM, Stelian Ionescu <sionescu-ct8TMaJCiNI< at >public.gmane.org> wrote: