gmane.lisp.flexi-streams.devel

substitution-char does not suppressexternal-format-encoding-error

Dmitriy Ivanov — 2012-01-21T12:06:17

Hello folks,

I have bumped into the following error while playing with Hunchentoot.
(It is originated from url-decoding GET parameters with
 *hunchentoot-default-external-format*.)

(let ((flex:*substitution-char* #\?))
  (flex:octets-to-string #(#xC1 #xC2 #xC3 #xC4) :external-format :utf-8))
=> "??"

(let ((flex:*substitution-char* #\?))
  (flex:octets-to-string #(#xC0 #xC1 #xC2 #xC3 #xC4) :external-format
:utf-8))
-> signals: This sequence can't be decoded using UTF-8 as it is too short.
1
octet missing at then end.

The reason is rather "simple": the decoder invokes the following chain of calls:
  compute-number-of-chars -> check-end -> signal-encoding-error

This contrasts to the most of decoder code, which directly calls
   recover-from-encoding-error
instead of
  signal-encoding-error.
--
Sincerely,
Dmitriy Ivanov
lisp.ystok.ru

unit test failure on sbcl

Anton Vodonosov — 2011-10-29T14:27:30

Hello.

When running asdf:test-op on the flexi-streams 1.0.7 with SBCL some failures occur.

The error message is 

Test (STRING=
      (FLEXI-STREAMS-TEST::OLD-OCTETS-TO-STRING
       FLEXI-STREAMS-TEST::OCTETS-VECTOR :EXTERNAL-FORMAT
       FLEXI-STREAMS-TEST::EXTERNAL-FORMAT)
      STRING) failed signalling error of type TYPE-ERROR: The value 0
                                                          is not of type
                                                            (MEMBER NIL T).

If I comment lines 341, 342 in test.lisp, the error does not occur.

I googled for that error and found a code snippet 
(http://paste.lisp.org/display/73334) which reproduces the
error:

(with-input-from-string (in "123456")
  (let ((stream (flex:make-flexi-stream in))                                                   
        (buffer (make-sequence 'string 3)))
    (read-sequence buffer stream)))

The code is not correct because it creates flexi-stream on top of character but not binary
stream. But the error message is the same, probably it might help you identify the 
reason why the tests fail.

Best regards,
- Anton

My open source libraries (aka "ediware")

Edi Weitz — 2011-08-31T15:11:30

[My apologies if you receive multiple copies of this email.]


Hi everybody!

As some of you will know, I'll start on a new job tomorrow.  This new
job won't involve much hacking, if at all, and thus it doesn't look
like I'll have a lot of time to maintain my open source libraries in
the near future.  I have no plans to suddenly disappear from the CL
world, but don't expect new releases of any of my libs any time soon.
(At least none published by me or on my server.)

Luckily, Hans Hübner - who already did most of the maintenance and
development work for Hunchentoot in the last two years or so - offered
to coordinate further development via github.  See his full
announcement at
<http://netzhansa.blogspot.com/2011/08/ediware-moving-to-github.html>.

I'll continue to read the mailing lists for my libs and I'm still
interested in fixing bugs you might find in the release tarballs
available on my web server.  However, I will likely not bother to
discuss or work on new features or compatibility code for
implementations other than LispWorks (which happens to be the one I'm
using).

Lastly, I hope to see a lot of you in Amsterdam
<http://weitz.de/eclm2011/> in October.  The number of registrations
so far has been pretty disappointing, but you still have three weeks
left to change your mind... :)

Take care,
Edi.

Flexi Streams fails to compile and install onMacPorts ECL

Andrew Pennebaker — 2011-03-01T22:12:50

Specs:

flexi-streams 1.0.7
Quicklisp 2010121400
ECL 11.1.1
MacPorts 1.9.2
Mac OS X 10.6.6
MacBook Pro 5,1

Trace:


...

;;;   Invoking external command:
;;;   /usr/bin/gcc-4.2 -o
/Users/andrew/.cache/common-lisp/ecl-11.1.1-macosx-x86/Users/andrew/quicklisp/dists/quicklisp/software/flexi-streams-1.0.7/mapping.fas
-L/Users/andrew/macports/lib/
/private/var/folders/N-/N-IxaaKuFae5ik1WKxV6wE+++TI/-Tmp-/eclinit87rR0W.o
/Users/andrew/.cache/common-lisp/ecl-11.1.1-macosx-x86/Users/andrew/quicklisp/dists/quicklisp/software/flexi-streams-1.0.7/mapping.o
-bundle -L/Users/andrew/macports/lib -arch x86_64
-L/Users/andrew/macports/lib -arch x86_64 -lecl -lm
In function COERCE, the value of variable is
        65533
which is not of expected type (INTEGER -128 127)

Available restarts:

1. (ABORT) ABORT
2. (TRY-RECOMPILING) Try recompiling ascii
3. (RETRY) Retry compiling component ("flexi-streams" "ascii").
4. (ACCEPT) Continue, treating compiling component ("flexi-streams" "ascii")
as having been successful.
5. (ABORT) Give up on "hunchentoot"
6. (RESTART-TOPLEVEL) Go back to Top-Level REPL.

Broken at SI:BYTECODES. [Evaluation of: (QUICKLISP-CLIENT:QUICKLOAD
"hunchentoot")]

Cheers,
Andrew Pennebaker
_______________________________________________
flexi-streams-devel mailing list
flexi-streams-devel< at >common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/flexi-streams-devel

gbk patch of flexi-streams.

jingtao xu — 2011-02-21T11:31:04

hi all

i have published my gbk(cp936,gb2312) encoding support for flexi-streams.
this patch passed all tests and use the best fit mapping table.
flexi-streams-devel< at >common-lisp.net

the git reposity url: https://github.com/jingtaozf/flexi-streams

Could weitz apply this patch to his official site?

With Best Regards.

jingtao.
_______________________________________________
flexi-streams-devel mailing list
flexi-streams-devel< at >common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/flexi-streams-devel

gbk patch to flexi-streams.

Xu Jingtao — 2011-02-21T10:33:13

hi all:
I have put one patch of flexi-streams to github:
This patch has passed all tests and use the new encoding map table.
https://github.com/jingtaozf/flexi-streams

UNREAD-BYTE and PEEK-BYTE onIN-MEMORY-INPUT-STREAM

Daniel Oliveira — 2010-12-11T18:48:39

Hello, i'm using FLEXI-STREAMS and i have a problem, UNREAD-BYTE and
PEEK-BYTE aren't defined for IN-MEMORY-INPUT-STREAM's, is there a reason why
they can't be or why it doesn't make sense for them to be?
_______________________________________________
flexi-streams-devel mailing list
flexi-streams-devel< at >common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/flexi-streams-devel

Flexi-streams problem with SBCL 1.0.34

Fred Gibson — 2010-01-07T20:29:00

I found a problem with the flexi-streams:string-to-octets function.  I
was getting the following back-trace when using it:

(ironclad:make-hmac (string-octets key) :sha1)  from zs3:make-digester gives:

The value #(55 98 50 52 104 75 97 65 50 54 ...)
is not of type
 (SIMPLE-ARRAY (UNSIGNED-BYTE 8) (*)).
 [Condition of type TYPE-ERROR]

Restarts:
 0: [DEFAULT-DEBUGGER] Use default debugger.
 1: [ABORT] Return to SLIME's top level.
 2: [TERMINATE-THREAD] Terminate this thread (#<THREAD "repl-thread"
RUNNING {ACA40D1}>)

Backtrace:
 0: ((SB-PCL::FAST-METHOD SHARED-INITIALIZE :AFTER (IRONCLAD::HMAC
T)) #<unavailable argument> #<unavail..
    Locals:
      SB-DEBUG::ARG-0 = 10
      SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
      SB-DEBUG::ARG-2 = :<NOT-AVAILABLE>
      SB-DEBUG::ARG-3 = #<HMAC(SHA1) {F0DD561}>
      SB-DEBUG::ARG-4 = :<NOT-AVAILABLE>
 1: ((LAMBDA (SB-PCL::|.P0.| SB-PCL::|.P1.| SB-PCL::|.P2.|))
#<unavailable argument> #<unavailable argum..
    Locals:
      SB-DEBUG::ARG-0 = :<NOT-AVAILABLE>
      SB-DEBUG::ARG-1 = :<NOT-AVAILABLE>
      SB-DEBUG::ARG-2 = :<NOT-AVAILABLE>

zs3's string-octets uses flexi-streams:string-to-octets

The workaround I'm using for now is instead going to acl-compat's
string-to-octets function from acl-excl.lisp:

(defun string-to-octets (string &key (null-terminate t) (start 0)
                       end mb-vector make-mb-vector?
                       (external-format :default))
 "This function returns a lisp-usb8-vector and the number of bytes copied."
 (declare (ignore external-format))
 ;; The end parameter is different in ACL's lambda list, but this
 ;; variant lets us give an argument :end nil explicitly, and the
 ;; right thing will happen
 (unless end (setf end (length string)))
 (let* ((number-of-octets (if null-terminate (1+ (- end start))
                             (- end start)))
       (mb-vector (cond
                    ((and mb-vector (>= (length mb-vector) number-of-octets))
                     mb-vector)
                    ((or (not mb-vector) make-mb-vector?)
                     (make-array (list number-of-octets)
                                 :element-type '(unsigned-byte 8)
                                 :initial-element 0))
                    (t (error "Was given a vector of length ~A, ~
                               but needed at least length ~A."
                              (length mb-vector) number-of-octets)))))
  (declare (type (simple-array (unsigned-byte 8) (*)) mb-vector))
  (loop for from-index from start below end
     for to-index upfrom 0
     do (progn
          (setf (aref mb-vector to-index)
                (char-code (aref string from-index)))))
  (when null-terminate
    (setf (aref mb-vector (1- number-of-octets)) 0))
  (values mb-vector number-of-octets)))

which seems to work fine.

My best,

Fred Gibson

Founder / Software Developer
http://www.streamfocus.com

(c)2010 Organon Technologies LLC