gmane.lisp.common-lisp.implementation-hackers

Re: LOOP non-compliance

Juan Jose Garcia-Ripoll — 2012-04-09T10:33:56

On Mon, Apr 9, 2012 at 11:59 AM, Nikodemus Siivola <
nikodemus-/W93PBX4W+ENs9zYA2MzYQ< at >public.gmane.org> wrote:

[...]
 I'll give you a fair warning, though: this is probably the most hated

Wouldn't it make sense to make it a warning, just like the LOOP problem
that started this thread?

Juanjo

Re: LOOP non-compliance

Nikodemus Siivola — 2012-04-09T09:59:04

On 8 April 2012 20:46, Juan Jose Garcia-Ripoll
<juanjose.garciaripoll< at >googlemail.com> wrote:



The error occurs when a DEFCONSTANT is evaluated where the value is
not EQL to the old one: typically whenever you file-compile and then
load into the same session a DEFCONSTANT that is not a character,
symbol, or number.

Condition signaled is SB-EXT:DEFCONSTANT-UNEQL, with accessors
SB-EXT:DEFCONSTANT-UNEQL-NAME, -OLD-VALUE and -NEW-VALUE.

CONTINUE restart changes the value.

ABORT restart keeps the old value.

(Though those restarts aren't documented -- they're just what is in
place right now. USE-VALUE would be a good addition.)

I'll give you a fair warning, though: this is probably the most hated
feature of SBCL...

Cheers,

 -- nikodemus

_______________________________________________
Implementation-hackers mailing list
Implementation-hackers< at >common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/implementation-hackers

Re: LOOP non-compliance

Juan Jose Garcia-Ripoll — 2012-04-09T07:47:30

So, where do you propose to do it? It's not the new implementation of

Reading LOOP's specification "more carefully" will not introduce the
semantics you wish. Right now the order of statements is "variable
iteration" followed by "main statements" = groups of conditions plus
imperative statements. You want to move the conditions anywhere, but are
not specifying how this should behave in relation to the order of
evaluation of the variable initialization and iteration and the existing
imperative and conditions. So it is _not_ a reinterpretation of what there
is, but a new behavior that has to be specified fully, in relation to past
behavior and all the existing clauses.

Currently there is no forum to change the standard. The closest to it is
the CDR project http://cdr.eurolisp.org/ which documents desired
extensions. It is not just chatting on a mailing list but somebody
committing to writing how some new behavior is introduced. What I do not
like about it is that it is not possible to vote, but that is the closest
to introducing a change in a rational way in which various implementations
may agree.

I do not have time to do that.

Juanjo

Re: LOOP non-compliance

Stas Boukarev — 2012-04-08T22:01:59

So, where do you propose to do it? It's not the new implementation of
LOOP, it's what all implementations have been doing for years, until
somebody read the spec more carefully.

When there will be time?

Re: LOOP non-compliance

Juan Jose Garcia-Ripoll — 2012-04-08T21:54:03


Ok, let's do it, here, right now.

Now, seriously, do you expect to develop _in a mailing list_, the next LOOP
implementation? Perfectly compatible with the standard, the new semantics
is not disruptive, and with the same detail of specification?

No time for that right now, thanks.

Juanjo

Re: LOOP non-compliance

Stas Boukarev — 2012-04-08T18:34:22

Then let all implementations to agree on a change.

Re: LOOP non-compliance

Juan Jose Garcia-Ripoll — 2012-04-08T17:48:01


Having WHILE out of order is not just a syntactic matter, but it implies a
change of the semantic in LOOP. Such a change cannot be left to the taste
of a single implementation.

Juanjo

Re: LOOP non-compliance

Juan Jose Garcia-Ripoll — 2012-04-08T17:46:43

On Sun, Apr 8, 2012 at 7:39 PM, Nikodemus Siivola <
nikodemus-/W93PBX4W+ENs9zYA2MzYQ< at >public.gmane.org> wrote:



Sounds fair. What restarts does SBCL set up in this case? Is it a
correctable error?

Re: LOOP non-compliance

Nikodemus Siivola — 2012-04-08T17:39:00

To play the devil's advocate here for a second, if we agree to take
strict view of LOOP clause ordering, will others also take strict view
of non-EQL DEFCONSTANTs like this?

  (defconstant +unportable+ "foo")

Cheers,

 -- nikodemus

Re: LOOP non-compliance

Greg Pfeil — 2012-04-08T13:27:11

…

FWIW, CLISP does warn in this case. If it hadn’t, I might have (incorrectly) filed a bug against ECL rather than (correctly) against Alexandria.

I agree that implementations should at least warn in this case. Maybe just subclass STYLE-WARNING with NON-STANDARD-BEHAVIOR-WARNING.

Re: LOOP non-compliance

Nikodemus Siivola — 2012-04-08T08:49:33



I'm on both sides of the fence at once. When working on my own stuff,
I quite like the non-standard option. When working stuff mean to be
portable, I don't want to be bitten by it.

Cheers,

 -- Nikodemus

_______________________________________________
Implementation-hackers mailing list
Implementation-hackers< at >common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/implementation-hackers

Re: LOOP non-compliance

Stas Boukarev — 2012-04-08T08:44:39

I wish the opposite was true (standard or not), that all implementations
accepted (loop while ... for ..), it's such a pain to write LOOPs without this
construct.

LOOP non-compliance

Juan Jose Garcia-Ripoll — 2012-04-08T08:35:11

This is probably a silent cry in the woods and nobody will listen, but
could other implementations please warn users that there is a precise order
of statements in LOOP?

http://www.lispworks.com/documentation/HyperSpec/Body/m_loop.htm

There are a lot of libraries out there which are being developed by
implementations that do not care whether (LOOP WHILE ... FOR I = ...) is a
valid or not, and this is at least affecting users of one implementation
that does (ECL).

You might say that it is up to the users to learn the Standard, but in
practice implementations may create a false impression that certain things
are ok, specially when those deviations are not explicit by the
implementation.

Juanjo

Re: [pro] Declarations in compilers (feedback welcome)

Martin Simmons — 2011-12-30T16:29:05

I don't like this because it contradicts the CL spec:

"The meaning of a type declaration is equivalent to changing each reference to
a variable (var) within the scope of the declaration to (the typespec var),
changing each expression assigned to the variable (new-value) within the scope
of the declaration to (the typespec new-value), and executing (the typespec
var) at the moment the scope of the declaration is entered."

(from http://www.lispworks.com/documentation/HyperSpec/Body/d_type.htm).

In LispWorks, type declarations and THE forms have the same semantics and they
are checked when safety = 3 and debug = 3.  The reason for involving debug is
that the checking code can be large and relatively slow.

__Martin

Re: [pro] Declarations in compilers (feedback welcome)

Juan Jose Garcia-Ripoll — 2011-12-29T17:50:05


Precisely what I mean is that the current semantics is really inconvenient
for library writers. I also believe that this change can be introduced at
no cost for library maintainers because it effectively does not change the
semantics at the safety levels that code are typically compiled (0 or
default ones). Let me try to explain it further below.



The issue is not SPEED, it is safety. Safety need not be sacrificed to gain
speed. Moreover, the problem with this SAFETY vs SPEED thing is that it has
no granularity at all. It is a simplistic view of the world which assumes
that all code is the same.

Let me explain the situation with an ordinary library, say a regular
expression parser. Somebody who writes the library has to understand that
there are various types of routines or sections of code that she is going
to write:

2- Code that handles user input (strings, lists which might be malformed,
etc)
1- Code that handles internal data (structures that will not change, sealed
classes, lists of known lengths)
0- Small sections of code that handles internal data and needs speed

I would expect that only 0 should be compiled with SAFETY = 0, and
explicitly marked so. However, we also have 1 and 2, which typically 1 and
2 are going to coexist and sometimes appear intermixed in the same
function. Here one must either resort to high safety levels for everything,
or end up wrapping around different sections of code with (LOCALLY (...
UNSAFE ...) ...) declarations. This is not good in my opinion.

The problem is that we are implicitly advocating that SAFETY = 0 is good
for everything once the code is mature enough and you need speed, but such
level implies much more than believing type declarations, it typically
implies that the arguments to functions are not checked at all. Take (CAR
(THE CONS X)). There are multiple ways in which this CAR call can be
inlined. To get the optimal case in this situation where I am telling the
compiler "believe me, this is a CONS", I may be opening the can of worms by
lifting all type checks in other uses of CAR.

Why do I believe this does not really change the semantics in a significant
way? First of all because apart from SBCL's declaration policy there is not
an explicitly written commitment in any of the free (natively compiling)
common lisps out there about the meaning of optimization settings. In such
a panorama, I would guess that currently library maintainers more or less
follow the approach of lowering safety levels to 0 in speed-critical code
and leaving it at some default value that works with their favorite
implementation elsewhere. See for instance CL-PPCRE

(defvar *standard-optimize-settings*
  '(optimize speed (safety 0)(space 0) (debug 1) (compilation-speed 0)
#+:lispworks (hcl:fixnum-safety 0))...

From the user's point of view, the approach seems to be: if safety level is
zero, the compiler will make fast code, in default settings mode, I will
get type checking. The PCL also suggests this, and it seems to be a common
entry point for many new users. Moreover, users also cannot rely on CMUCL's
or SBCL's or ECL's type checking behavior for function arguments, because
they are not really standard, and manual type checking is required in most
libraries.

OTOH, if one comes up with a set of sensible settings that users may choose
from and which may be applicable throughout the library, without disrupting
the current behavior at SAFETY 0 or default, then  the cost of adoption is
zero.

I am just trying to figure out a non-disruptive way of choosing those
settings, documenting them (
http://ecls.sourceforge.net/new-manual/ch02.html#ansi.declarations.optimize),
and perhaps even sparking a debate about it, so that there may be some more
uniformity throughout implementations.

Cheers,

Juanjo

Re: Declarations in compilers (feedback welcome)

Nikodemus Siivola — 2011-12-29T15:08:20

On 29 December 2011 16:04, Juan Jose Garcia-Ripoll
<juanjose.garciaripoll< at >googlemail.com> wrote:


Fair enough. SBCL disagrees, but it and CMUCL stand apart from most
implementations when it comes to handling of types.


Actually, SBCL /should/ generate the check, unless you are using the
interpreter. If it didn't then I'm guessing the Ubuntu version is an
old one:

CL-USER> (defun foo (x) x)
CL-USER> (foo (the fixnum t))
; in: FOO (THE FIXNUM T)
;     (THE FIXNUM T)
;
; caught WARNING:
;   Constant T conflicts with its asserted type FIXNUM.
;   See also:
;     The SBCL Manual, Node "Handling of Types"
;
; compilation unit finished
;   caught 1 WARNING condition
; Evaluation aborted on #<SIMPLE-TYPE-ERROR expected-type: FIXNUM datum: T>.

...plus entry to debugger is the expected behaviour.

Both THE generates and and assignment to a variable whose type has
been declared generates a identical cast node in SBCL's IR.

Cheers,

 -- Nikodemus

_______________________________________________
Implementation-hackers mailing list
Implementation-hackers< at >common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/implementation-hackers

Re: Declarations in compilers (feedback welcome)

Juan Jose Garcia-Ripoll — 2011-12-29T14:04:48

On Thu, Dec 29, 2011 at 1:33 PM, Nikodemus Siivola <
nikodemus-/W93PBX4W+ENs9zYA2MzYQ< at >public.gmane.org> wrote:



I believe there can be a compromise between safety and speed. There are
many macros and user code that can perform assertions about the code that
the compiler will never be able to, and it is in my opinion unfortunate
that all the safety checks have to be removed to take full advantage of
those.

I also understand that some of the type checks are cheap, specially if the
compiler is allowed to "simplify" them, as SBCL does for SAFETY=1, but the
result is code bloat. Lots of avoidable checks, branching and error
messages that we could do without, without actually sacrificing safety.
That does not seem like a bad case for something in between both extremes.

Conceptually, in the model above, I do not see the THE as a loophole, but
rather as two different things: variable declarations = type checked
assignments, value declarations = compiler hints. For instance, if I invoke
a function with a THE argument, SBCL will not generate a check: (FOO (THE
FIXNUM X)) is just (FOO X), am I wrong? (I just checked in Ubuntu's SBCL)
Then in that sense THE does not really make much sense at all, because the
type checks are introduced by assignments to variables, not by this special
form.

Juanjo

Re: Declarations in compilers (feedback welcome)

Nikodemus Siivola — 2011-12-29T12:33:27

On 29 December 2011 12:24, Juan Jose Garcia-Ripoll
<juanjose.garciaripoll< at >googlemail.com> wrote:


This is correct, but incomplete. At SAFETY 1 SBCL will weaken complex
assertions: eg.

  (OR (INTEGER 0 2) (INTEGER 7 10))

will be simplified to a range check for (INTEGER 0 10). At SAFETY 2 no
types are weakened. At SAFETY 3 all the extra bells and whistles
required by ANSI come into play.

CMUCL's approach is very similar to SBCL's, but IIRC the policy on
weakening assertions is a bit different.


Actual cost of assertions (for SBCL generated code at least) is fairly
small. They should for the most part be branches which the static
branch prediction model gets right every time.


I somewhat dislike making THE a loophole -- IMO it complicates the
mental model necessary to understand how things work, especially if it
works differently at a specific SAFETY level. SBCL has
SB-EXT:TRULY-THE for just this purpose, which is roughly equivalent
to:

  (defmacro truly-the (type values)
    `(flet ((the-values () ,values))
       (declare (optimize (safety 0)))
       (the ,type (the-values))))

CMUCL has the equivalent as EXT:TRULY-THE. You may want to consider
something like that as well.

I know that when I write (UNSAFE-FUN-THAT-CHECKS-NOTHING (THE FIXNUM
X)) I intend the THE as an assertion. Granted, most of the code I
write is intended for SBCL-only consumption, so this is probably a
moot point. Still, loading bunch of stuff from Quicklisp and
instrumenting the compiler to see how often THE's like that occur
might be instructive.

At the end, as long as SAFETY 0 = trust everything blindly and SAFETY
3 = check everything, I think you're well within the bounds of custom
and sanity if you choose to make SAFETY 1 a bit magical.


This is actually a major point for us. Because SBCL open codes /
partial-evaluates things rather agressively, and has a fairly
extensive derivation machinery, in idiomatic Lisp code with type
declarations type checks mostly occur only for function arguments,
return values, and iteration variables -- and the cost of those type
checks if trivial for the most part. What sometimes makes them look
more expensive then they actually are is the suboptimal representation
selection they cause.


Yes, but if you proclaim FOO to return a fixnum before compiling it,
then FOO will take care of that assertion. (Trusting proclamations
made after a function has been compiled is considered a long-standing
bug, not a feature.)

Cheers,

 -- Nikodemus

_______________________________________________
Implementation-hackers mailing list
Implementation-hackers< at >common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/implementation-hackers

Declarations in compilers (feedback welcome)

Juan Jose Garcia-Ripoll — 2011-12-29T10:24:37

After struggling mentally with this for a few weeks, I would like to have
some consultation before I introduce some changes in ECL -- not that I
expect many users here, but at least some implementor-fellows and power
users of other implementations.

My concerns right now relate to how declarations should be used by a
compiler, and in particular how declarations interact with SAFETY levels.
Please correct me if I am wrong, but I have seen more or less the following
approaches

[a]- Most implementations blindly believe declarations below a certain
safety level. Above it, they seem more or less useless.

[b]- SBCL takes declarations (and THE) as type assertions. For instance, in
(LET ((Y (FOO X))) (DECLARE (FIXNUM Y))) ...) the assignment to Y would be
checked to be a FIXNUM. This means the type declaration is actually
enforced and believed and only at SAFETY 0 the checks are dropped (*)

In both cases one ends up with a model in which in order to truly believe a
declaration and have no extra burden (assertions), one has to drop to
SAFETY 0 in all code that is involved with it, which is a mess, because it
might inadvertently affect other parts of the code. It is for this reason
that I am considering an alternative model for ECL which would grade safety
as follows

- Type declarations are always believed
- SAFETY >= 1 adds type checks to enforce them.
- SAFETY = 0, no checks.
- SAFETY = 1, the special form THE or additional proclamations on the
functions can be used to deactivate the check. As in (LET ((Y (THE FIXNUM
(FOO X))) ...)

This would allow one to keep most code safe, while deactivating some checks
when they are really known to be true (**). Do you think this is
useful/useless? The problem I see with this approach is that all code
around is written for model [a] or [b], but I could not come up with
something more sensible so far.

Juanjo

(*) Actually the checks are also deactivated when SBCL can infer the type
of the value that is assigned to Y. This is somewhat contradictory, because
(SETF Y (THE FIXNUM (FOO X))) would still generate a check, but proclaiming
FOO to return a FIXNUM would completely bypass the check.

(**) Yes, indeed I know that LOCALLY exists for a reason, but it does more
than THE. For instance, if I (LOCALLY (DECLARE (SAFETY 0)) (THE FIXNUM (FOO
(SLOT-ACCESSOR X)))... this influences the safety of the code that accesses
a structure, which is not good.

P.S.: Thanks to Paul Khuong for pointing out that SBCL behaves differently
w.r.t. declarations.

Re: problems with DEFAULT-SPECIAL-BINDINGS

Juan Jose Garcia-Ripoll — 2010-04-03T14:00:40

_______________________________________________
Implementation-hackers mailing list
Implementation-hackers-F1HGIaG5STRyXAeb93iumQ< at >public.gmane.org
http://common-lisp.net/cgi-bin/mailman/listinfo/implementation-hackers

problems with DEFAULT-SPECIAL-BINDINGS

Tobias C. Rittweiler — 2010-04-03T11:01:21

Most implementations that provide multi-threading also provide a
variable which contains an association lists for variable bindings to be
established in new threads.

I wanted to introduce the same to SBCL.

However, Gábor Melis made me aware of the following issue:


One use of *DEFAULT-SPECIAL-BINDINGS* is for libraries to establish
thread-local variable bindings to new threads by pushing onto
*DEFAULT-SPECIAL-BINDINGS*.

Howsoever, that will only affect new threads, not already running
threads.

That's a big problem for the following scenario:

Let's say your Lisp development environment is multi-threaded and allows
you to run multiple listeners at the same time. Each listener is run in
its own thread.

Now, you opened a few listeners, then loaded in this library which
pushes onto *DEFAULT-SPECIAL-BINDINGS*. Now all running listeners, when
the user uses them to call into that library, will operate on the _same_
global binding. 

Hilarity ensues.


Do you have ideas on how to provide a mechanism that handles this
scenario reliably?

  -T.

gmane.lisp.common-lisp.implementation-hackers

Re: LOOP non-compliance

Re: LOOP non-compliance

Re: LOOP non-compliance

Re: LOOP non-compliance

Re: LOOP non-compliance

Re: LOOP non-compliance

Re: LOOP non-compliance

Re: LOOP non-compliance

Re: LOOP non-compliance

Re: LOOP non-compliance

Re: LOOP non-compliance

Re: LOOP non-compliance

LOOP non-compliance

Re: [pro] Declarations in compilers (feedback welcome)

Re: [pro] Declarations in compilers (feedback welcome)

Re: Declarations in compilers (feedback welcome)

Re: Declarations in compilers (feedback welcome)

Re: Declarations in compilers (feedback welcome)

Declarations in compilers (feedback welcome)

Re: problems with *DEFAULT-SPECIAL-BINDINGS*

problems with *DEFAULT-SPECIAL-BINDINGS*

Re: problems with DEFAULT-SPECIAL-BINDINGS

problems with DEFAULT-SPECIAL-BINDINGS