http://blog.gmane.org/gmane.lisp.steel-bank.devel hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16756 <pre>On Fri, May 25, 2012 at 1:18 AM, Nikodemus Siivola &lt;nikodemus&lt; at &gt;random-state.net&gt; wrote: Yes, it's not there during cross-compilation. In DEFINE-ARRAY-DISPATCH if you replace for (typecode specifier primitive-type-name) in %%saetp-info%% with for saetp across sb!vm:*specialized-array-element-type-properties* for typecode = (sb!vm:saetp-typecode saetp) for specifier = (sb!vm:saetp-specifier saetp) for primitive-type-name = (sb!vm:saetp-primitive-type-name saetp) then make.sh will fail at ; x-compiling (DEFMACRO DEFINE-ARRAY-DISPATCH ...) with ; Undefined variable: ; SB!VM:*SPECIALIZED-ARRAY-ELEMENT-TYPE-PROPERTIES* Well remember the dispatch is only being done on the INTO array. This is like a partial open-coding, but it's still very far behind the real open-coded MAP-INTO. When all arrays are declared (the open-coder appears to require that), the open-coded MAP-INTO gives a 10x speed increase. (Code attached.) There's no obstacle to writing another macro on top of DEFINE-ARRAY-DISPATCH which dispatches on two arrays, generating (expt (length sb-vm:*specialized-array-element-type-properties*) 2) number of functions. The only question is where to draw the line on the speed/space trade-off. ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ Sbcl-devel mailing list Sbcl-devel&lt; at &gt;lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sbcl-devel </pre> James M. Lawrence 2012-05-25T09:39:38 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16755 <pre> +;;; Store some saetp fields for DEFINE-ARRAY-DISPATCH since +;;; sb!vm:*specialized-array-element-type-properties* is not always +;;; available. This sounds odd. Do you mean it's not there during some stage of the build? Otherwise the code is lovely. Almost certainly, since it means less open coding in user-code: the MAP-INTO transform can then be made conditional on eg. SPEED=3, or INLINE declaration. Putting functions in the vector means you get full call overhead, and disables local call analysis -- so it's not entirely surprising, even if the dispatch worked fast. We /could/ make big typecases with array-types use binary search on the widetag... I think most sequence functions would benefit from it. Aside from SUBSEQ and FILL we're currently relying excessively on inlining to make them fast -- fast out-of-line implementations would be better, IMO. Unless someone gets there first, I plan on merging these over the weekend. Cheers,  -- Nikodemus ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Sbcl-devel mailing list Sbcl-devel&lt; at &gt;lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sbcl-devel </pre> Nikodemus Siivola 2012-05-25T05:18:33 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16754 <pre>I would like to optimize some (apply #'map-into ..) calls where the sequence(s) passed are not created by me. That is, optimize the non-open-coded MAP-INTO. I gathered the pieces of the widetag dispatching code for VECTOR-SUBSEQ* and wrote a general macro for creating dispatch tables. Using the first parameter as the specialized array, DEFINE-ARRAY-DISPATCH defines each specializing function with a corresponding type declaration inside. VECTOR-SUBSEQ* may now be written as: (define-array-dispatch vector-subseq-dispatch (array start end) (declare (optimize speed (safety 0))) (declare (type index start end)) (subseq array start end)) (defun vector-subseq* (sequence start end) (declare (type vector sequence)) (declare (type index start) (type (or null index) end) (optimize speed)) (with-array-data ((data sequence) (start start) (end end) :check-fill-pointer t :force-inline t) (vector-subseq-dispatch data start end))) To slightly complicate matters, the current MAP-INTO is kludgy. With that kludginess fixed[1], a basic MAP-INTO benchmark drops from 385ms to 227ms. When we add widetag dispatching[2][3] on top of that fix, it goes from 227ms to 110ms. However it's not clear whether it is appropriate to do this optimization inside SBCL since a time/space trade-off is involved. Is an 86K core size increase (uncompressed) worth it? This is only for MAP-INTO. Of course, if a user wants performance then he should be using declarations with the open-coded MAP-INTO. But there are still cases where declarations can't easily be made, my own situation being one of them. It would be nice to have something like (defmacro with-declared-array-type (array &amp;body body) (check-type array symbol) `(typecase ,array ,&lt; at &gt;(loop :for saetp :across sb-vm:*specialized-array-element-type-properties* :collect `((simple-array ,(sb-vm:saetp-specifier saetp)) ,&lt; at &gt;body)) (otherwise ,&lt; at &gt;body))) available in userland. This would make it easier to do such optimizations without depending upon internals. Ideally it would have constant lookup, although the linear search with TYPEP is quick enough for most purposes. (Out of curiosity I implemented WITH-DECLARED-ARRAY-TYPE using a stack-allocated vector of FLET functions, but this was slower than TYPECASE even for the worst-case TYPEP search. A fast WITH-DECLARED-ARRAY-TYPE seems possible in principle, though it would presumably require new special operator(s) and/or magic.) Getting back to SBCL innards, considering that SUBSEQ and FILL already use widetag dispatch there may be another place which would benefit from it. If so then [2] will help. If not then it may be needless abstraction. [1] 0001-fix-MAP-INTO-performance.patch -- same as bug #1001043 [2] 0002-automate-widetag-dispatching.patch [3] 0003-widetag-dispatch-for-MAP-INTO.patch -- needs [1] ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ Sbcl-devel mailing list Sbcl-devel&lt; at &gt;lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sbcl-devel </pre> James M. Lawrence 2012-05-24T09:10:19 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16753 <pre> I think what is happening here is more of the same as last month. When cleverly deriving that the arithmetic operations mean that some of the bits in bignums will not affect the answer, and hence that the operation can be performed with fewer bits (and signed arithmetic), it is important that all the operands be converted -- otherwise a high bit in one operand will be in the same place as the infinite sequence of sign bits in the other, and the wrong answer will ensue. In FOO, above, the combination should be changed to (logand (sb-c::mask-signed-field 63 8459622733968096971) (sb-c::mask-signed-field 63 x)) which gives the right answer, but unfortunately the second operand is not so converted in the case above. The BAR (lognand) case is I think similar; the LOGNOT is compiled to a machine instruction, but unfortunately the previous LOGAND has returned a bignum, so completely the wrong thing occurs. To fix this, what needs to happen in CUT-TO-WIDTH is some more surgery: whenever a node is not either a constant or a call to a modular function, and the compiler cannot prove it is already of the appropriate type, it needs to be wrapped in the appropriate (logand ...) or (mask-signed-field ...) to ensure that the arguments are cut uniformly. The potential for hilarious infinite recursion has already been spotted by someone (Alexey?) for the logand case; a similar clause for mask-signed-field would probably make sure everything is happy. But I'm not sure how to perform the wrapping -- is there an IR1 surgeon in the house? Cheers, Christophe ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ </pre> Christophe Rhodes 2012-05-23T15:06:41 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16752 <pre>Hi, * (lisp-implementation-version) "1.0.57.15-35f68f4" * (defun foo (x) (declare (optimize (space 2)) (type (integer 12417236377505266230 12417274239874990070) x)) (logand 8459622733968096971 x)) FOO * (foo 12417237222845306758) 11836165733894624898 ;; &lt;-- incorrect * (logand 8459622733968096971 12417237222845306758) 2612793697039849090 * (defun bar (x) (declare (type (integer 22965360520649903105 22965361070405717069) x)) (lognand 6936474818856893141 x)) BAR * (bar 22965361070405716988) #&lt;unknown immediate object, lowtag=#b1, widetag=#x41 {FFFFFFEFFD4D3D41}&gt; (More random-integer testing.) </pre> Eric Marsden 2012-05-23T12:01:08 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16751 <pre> ...stuff... Amusingly, it turned out that the TYPEP consing came from NLX value cells as well. Rudimentary escape analysis is enough to sort this out. I already committed the SIGNAL improvements, but would much appreciate it if someone could find the time to review this one before I commit it. I keep having the feeling I'm missing something obvious. Non-loadable patch attached, apply on top of 9bc5da72887b15eb83500e16f05c3e42835476a3 or later. With that setup, the example above should run without consing. Cheers,  -- Nikodemus ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ Sbcl-devel mailing list Sbcl-devel&lt; at &gt;lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sbcl-devel </pre> Nikodemus Siivola 2012-05-22T19:43:14 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16750 <pre> There appears to be no cc in your path. Only sb-concurrency should fail the tests build. </pre> Stas Boukarev 2012-05-22T14:59:11 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16749 <pre>Sorry for not catching this earlier, but there seem to be build issues on Windows in this release. The build concludes with some failed contribs (notably bsd-sockets and posix), but I haven't had time to bisect or track down the root cause yet. The complete build log is here: https://elliottslaughter.com/files/build-sbcl-1.0.57.txt If I get time, I'll try to look into this more later. On Sun, May 13, 2012 at 7:02 PM, Juho Snellman &lt;jsnell&lt; at &gt;iki.fi&gt; wrote: </pre> Elliott Slaughter 2012-05-22T07:19:19 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16748 <pre> ERROR was actually already smart about this -- it didn't do till it knew it had to invoke the debugger, which was actually the reason I ended up doing this change: I wanted to have the hint available when doing (handler-bind ((error #'invoke-debugger)) ...) Anyways, I was too optimistic. It's still pretty consy: - Value cell for the nlx (GO :AGAIN) is heap allocated. Should be safe to elide for exists from DX lambdas, though, and that one is a DX lambda... - SIGNAL conses a lot -- most of it needlessly when *BREAK-ON-SIGNALS* is NIL. Conditionalizing the RESTART-CASE on that helps. After those (TYPEP CONDITION (CAR HANDLER)) seems to be the last source of consing in SIGNAL, aside from the &amp;REST list allocation which doesn't matter here. Attached loadable patch does these things. Not non-consing yet, but a lot better: Evaluation took: 0.247 seconds of real time 0.244681 seconds of total run time (0.243418 user, 0.001263 system) [ Run times consist of 0.007 seconds GC time, and 0.238 seconds non-GC time. ] 99.19% CPU 637,329,264 processor cycles 15,990,784 bytes consed versus the old Evaluation took: 0.517 seconds of real time 0.514088 seconds of total run time (0.464597 user, 0.049491 system) [ Run times consist of 0.011 seconds GC time, and 0.504 seconds non-GC time. ] 99.42% CPU 1,338,492,272 processor cycles 272,010,416 bytes consed Cheers, -- nikodemus ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ Sbcl-devel mailing list Sbcl-devel&lt; at &gt;lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sbcl-devel </pre> Nikodemus Siivola 2012-05-21T20:16:44 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16747 <pre> Still in my "queue", which looks more like a spread every day... What's the pull-url? I'll try to get at least some of these in this month. Cheers, -- nikodemus ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ </pre> Nikodemus Siivola 2012-05-21T18:49:41 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16746 <pre>I would expect so, yes. ...let me know if it doesn't. Ns Sent from my phone - apologies for substandard spelling and formatting. On May 21, 2012 4:49 PM, "Tobias C Rittweiler" &lt;tcr&lt; at &gt;freebits.de&gt; wrote: ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ Sbcl-devel mailing list Sbcl-devel&lt; at &gt;lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sbcl-devel </pre> Nikodemus Siivola 2012-05-21T16:00:24 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16745 <pre>Nikodemus, any news on this? i used to keep on rebasing my changes but it starts to feel as wasted time... the most recent version, modulo the last conflicts, is available here: http://dwim.hu/gitweb/gitweb.cgi?p=sbcl;a=summary </pre> Attila Lendvai 2012-05-21T15:57:34 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16744 <pre>In article &lt;E1SWLPN-0000Yo-Kv&lt; at &gt;sfs-ml-1.v29.ch3.sourceforge.com&gt;, "Nikodemus Siivola" &lt;demoss&lt; at &gt;users.sourceforge.net&gt; wrote: Thank you! Does it mean that (define-condition once-again () ()) (prog ((+once-again+ (make-condition 'once-again)) (n 1000000)) :again (handler-bind ((once-again #'(lambda (c) (declare (ignore c)) (go :again)))) (unless (zerop (decf n)) (error +once-again+)))) now runs in nearly constant space? T ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ </pre> Tobias C Rittweiler 2012-05-21T13:44:19 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16743 <pre> +1 Cheers, -- nikodemus ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ </pre> Nikodemus Siivola 2012-05-17T07:19:23 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16742 <pre> I'm always annoyed by messages like: Implicitly creating new generic function FOO. scrolling by in my slime window. WTF is FOO? Well, the problem is that FOO is usually in the then-current package and so the package identifier isn't printed out. The following patch causes the package name to be printed in the STYLE-WARNING. Any objections to seeing this in the tree post-freeze? thanks, Cyrus diff --git a/src/code/condition.lisp b/src/code/condition.lisp index 3e7e84e..d0b7013 100644 --- a/src/code/condition.lisp +++ b/src/code/condition.lisp &lt; at &gt;&lt; at &gt; -1004,8 +1004,9 &lt; at &gt;&lt; at &gt; (define-condition implicit-generic-function-warning (style-warning) ((name :initarg :name :reader implicit-generic-function-name)) (:report (lambda (condition stream) - (format stream "~&lt; at &gt;&lt;Implicitly creating new generic function ~S.~:&lt; at &gt;&gt;" - (implicit-generic-function-name condition))))) + (let ((*package* (find-package :keyword))) + (format stream "~&lt; at &gt;&lt;Implicitly creating new generic function ~S.~:&lt; at &gt;&gt;" + (implicit-generic-function-name condition)))))) (define-condition extension-failure (reference-condition simple-error) ()) ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ </pre> Cyrus Harmon 2012-05-17T04:34:10 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16741 <pre> For me, SBCL warns me when it removes unreachable code. This usually turns up bugs in my typing or logic or paren-grouping. I guess I was expecting to see some indication that dead code was removed, too. If I have dead code, why would I want I there? And, if I do want it there, I'd like to know if it isn't. But, maybe I'm being too optimistic. Maybe everything would be warning me everywhere... like the "else" clause of an IF when expanded from an WHEN macro. </pre> Patrick Stein 2012-05-16T12:23:46 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16740 <pre>[...] [...] [...] In both cases, the code depends on undefined behaviour (I can only imagine that you had functions that simply read variables without using their value in an attempt to trigger unbound variable conditions). CL's default level for all optimization levels is 1; for SBCL, that level means that we don't crash on undefined behaviour, and instead signal conditions. In these cases, the potential error has been optimised away: there's no crash to replace with a condition anymore. I, and many other users I suppose, would be peeved it SBCL disabled simple optimisations like dead code elimination just to consistently signal a condition in the face of undefined behaviour. Optimisations in SBCL guarantee that you'll get the result and side effects your code defines, and, when safety isn't 0, conditions instead of random behaviour if runtime errors would happen. There aren't many potential optimisations one can perform if they wish to signal exactly the same conditions as a naïve interpreter. Paul Khuong ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ </pre> Paul Khuong 2012-05-16T07:37:33 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16739 <pre> I see what you mean by using the return of the result. I stand corrected. A while ago I was writing an assembler for an embedded architecture where common lisp read the opcodes in the form of sexps (as actual common lisp). If I used symbols (in the assembler's DSL, but also in the lisp variable namespace) that weren't defined, I'd handle the unbound-variable conditions and poke them into a symbol table for later assignment to addresses when the opcodes were actually assembled. It simplified not having to write more structure around the opcode list to predefine the variables or walk over the opcode list try and pick out the symbols that were variables in the assembly DSL (which was hard). I'm not claiming that I chose a good way to do that project, but it was a place where I had to modify SBCL's safety to 3 to get the right behavior. Up until then, it didn't even occurr to me that SBCL's safety wasn't already 3. Later, -pete ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Sbcl-devel mailing list Sbcl-devel&lt; at &gt;lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sbcl-devel </pre> Peter Keller 2012-05-16T04:27:14 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16738 <pre> If you were to sped up the computation that would imply that you care about the result. And handler-bind would only slow things down. (defun does-not-absorb () (let ((ret 'no-error)) (list (handler-case (let ((y 0)) (/ y)) (division-by-zero (c) (setf ret c))) ret))) By return the result, the division is not elided and the error is signalled. I can't even contrive any reasons for this to ever be useful. safety 1 in SBCL is very reasonable, unless your code is contorted. </pre> Stas Boukarev 2012-05-16T03:43:01 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16737 <pre> If one were writing a video game where one didn't want to check for a division by zero to speed up the computation (just handling the condition in a handler-bind when it happens) that would be a plenty good reason to want the divide-by-zero condition. I've often been surprised by the times that SBCL's default safety settings don't cause certain conditions, like unbound-variable conditions and such, to be signaled at the appropriate times. I know now to set saftey to 3 to get safe code, but it was a surprising road to find that out. I figured I would just get safe code by default. Thank you. -pete ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Sbcl-devel mailing list Sbcl-devel&lt; at &gt;lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sbcl-devel </pre> Peter Keller 2012-05-16T03:35:45 http://permalink.gmane.org/gmane.lisp.steel-bank.devel/16736 <pre> It doesn't have side-effects, except for the error, which i don't think anyone would desire. If you want to check everything, run with (safety 3), which is the definition of safe code by CLHS. </pre> Stas Boukarev 2012-05-16T03:02:49 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.lisp.steel-bank.devel