gmane.lisp.babel.devel

error downloading babel

2012-03-23T15:35:09

Hello,

I encounter a problem trying to download babel using quicklisp. 

I am on debian squeeze and I use the last version of sbcl 1.0.55 that I
downloaded 5mns ago from the website, together with a fresh install of
quicklisp. I can reproduce the error using the version of sbcl 1.0.40
in the debian repositories. 


The relevant part of the log which is attached to this message seems to be
* (ql:quickload "babel")
To load "babel":
  Load 1 ASDF system:
    babel
; Loading "babel"

; file: /home/harven/.quicklisp/dists/quicklisp/software/babel-20120208-git/src/enc-unicode.lisp
; in: DEFINE-UTF-16 :UTF-16
;     (BABEL-ENCODINGS::DEFINE-UTF-16 :UTF-16)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UTF-16 :UTF-16))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).


A similar error has been reported on github 
https://github.com/cl-babel/babel/issues/9
where it was suggested to run the command
(alexandria:format-symbol t '#:~a-code-point-counter "UTF16") 
Here is what I get.

* (ql:quickload "alexandria")
To load "alexandria":
  Load 1 ASDF system:
    alexandria
; Loading "alexandria"

("alexandria")
* (alexandria:format-symbol t '#:~a-code-point-counter "UTF16") 

debugger invoked on a SB-KERNEL:CASE-FAILURE in thread
#<THREAD "initial thread" RUNNING {AB2C889}>:
  #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
  Wanted one of (STRING SIMPLE-STRING).

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

(SB-KERNEL:CASE-FAILURE
 ETYPECASE
 #:~A-CODE-POINT-COUNTER
 (STRING SIMPLE-STRING))
0] 0

Any help ?

* (ql:quickload "babel")
To load "babel":
  Load 1 ASDF system:
    babel
; Loading "babel"

; file: /home/harven/.quicklisp/dists/quicklisp/software/babel-20120208-git/src/enc-unicode.lisp
; in: DEFINE-UTF-16 :UTF-16
;     (BABEL-ENCODINGS::DEFINE-UTF-16 :UTF-16)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UTF-16 :UTF-16))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UTF-16 :UTF-16LE
;     (BABEL-ENCODINGS::DEFINE-UTF-16 :UTF-16LE :LE)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UTF-16 :UTF-16LE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UTF-16 :UTF-16BE
;     (BABEL-ENCODINGS::DEFINE-UTF-16 :UTF-16BE :BE)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UTF-16 :UTF-16BE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UTF-32
;     (BABEL-ENCODINGS::DEFINE-UCS :UTF-32 4)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UTF-32 ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UTF-32LE
;     (BABEL-ENCODINGS::DEFINE-UCS :UTF-32LE 4 :LE)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UTF-32LE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UTF-32BE
;     (BABEL-ENCODINGS::DEFINE-UCS :UTF-32BE 4 :BE)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UTF-32BE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UCS-2
;     (BABEL-ENCODINGS::DEFINE-UCS :UCS-2 2 NIL 65536)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UCS-2 ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UCS-2LE
;     (BABEL-ENCODINGS::DEFINE-UCS :UCS-2LE 2 :LE 65536)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UCS-2LE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

; in: DEFINE-UCS :UCS-2BE
;     (BABEL-ENCODINGS::DEFINE-UCS :UCS-2BE 2 :BE 65536)
; 
; caught ERROR:
;   (during macroexpansion of (DEFINE-UCS :UCS-2BE ...))
;   #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
;   Wanted one of (STRING SIMPLE-STRING).

debugger invoked on a ASDF:COMPILE-ERROR in thread
#<THREAD "initial thread" RUNNING {AB2C889}>:
  Error while invoking #<COMPILE-OP (:VERBOSE NIL) {B72ECF9}> on
  #<CL-SOURCE-FILE "babel" "src" "enc-unicode">

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RETRY ] Retry compiling #<CL-SOURCE-FILE "babel" "src" "enc-unicode">.
  1: [ACCEPT] Continue, treating
              compiling #<CL-SOURCE-FILE "babel" "src" "enc-unicode"> as having
              been successful.
  2: [ABORT ] Give up on "babel"
  3:          Exit debugger, returning to top level.

((SB-PCL::FAST-METHOD ASDF:PERFORM (ASDF:COMPILE-OP ASDF:CL-SOURCE-FILE))
 #<unavailable argument>
 #<unavailable argument>
 #<ASDF:COMPILE-OP (:VERBOSE NIL) {B72ECF9}>
 #<ASDF:CL-SOURCE-FILE "babel" "src" "enc-unicode">)
0] 

Sincerely,

Problem with gbk-map.lisp

Luís Oliveira — 2012-01-02T22:05:53

Hello Wenpeng,

I missed an obvious problem with your patch, it depends on Lisps
loading gbk-map.lisp using the UTF-8 encoding. AFAIK, there's no good
way to portably enforce that via ASDF, so I've converted the file to
an ASCII representation. It's not as pretty as the previous version,
but it seems to work.

Cheers,

Unit tests failures on different lisps

Anton Vodonosov — 2011-12-28T21:21:34

Hello.

I am running tests of some most often Quicklisp-downloaded libraries, including babel.

Babel tests have different number of failures/errors on different Lisps (about 8, 9 or 5).

You may find the results here: http://common-lisp.net/project/cl-test-grid/pivot_ql-lib_lisp.html

Clicking the ok/fail status refer to the library test logs where you may find what failures
occurred.

Best regards,
- Anton

Submit a GBK patch

levin — 2011-12-21T11:21:33

HI,all

Days ago, I found this project babel, and tried to use it in my project, as
I'm Chinese,
I found it doesn't support GBK encoding in babel, so I wrote a patch for
babel to make
it support GBK.

I just used common lisp for not a long time, so if there was something not
good enough, please let me know, I also hope you can accept this patch or
help me
modify it to make babel support GBK, so we can use it freely to process
Chinese text.

Thanks.

Few fixes for ucs-2 and utf-32

Dmitry Ignatiev — 2011-10-27T13:46:25

Hi there.

There were some BOM-related bugs in unicode decoders.

patch attached
_______________________________________________
babel-devel mailing list
babel-devel< at >common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/babel-devel

patch for cp1252

Nicolas Martyanoff — 2011-04-23T14:50:54

Hi,

I added support for the cp1252 encoding:

Sat Apr 23 16:41:22 CEST 2011  khaelin< at >gmail.com
  * add support for cp1252 encoding
diff -rN -u old-babel/babel.asd new-babel/babel.asd
--- old-babel/babel.asd2011-04-23 16:49:25.088659055 +0200
+++ new-babel/babel.asd2011-04-23 16:49:25.091992342 +0200
< at >< at > -41,6 +41,7 < at >< at >
      (:file "enc-iso-8859")
      (:file "enc-unicode")
      (:file "enc-cp1251")
+     (:file "enc-cp1252")
      (:file "jpn-table")
      (:file "enc-jpn")
      (:file "external-format")
diff -rN -u old-babel/src/enc-cp1252.lisp new-babel/src/enc-cp1252.lisp
--- old-babel/src/enc-cp1252.lisp1970-01-01 01:00:00.000000000 +0100
+++ new-babel/src/enc-cp1252.lisp2011-04-23 16:49:25.098658916 +0200
< at >< at > -0,0 +1,81 < at >< at >
+;;;; -*- Mode: lisp; indent-tabs-mode: nil -*-
+;;;
+;;; enc-cp1252.lisp --- Implementation of the CP1252 character encoding.
+;;;
+;;; Copyright (C) 2011, Nicolas Martyanoff
+;;;
+;;; Permission is hereby granted, free of charge, to any person
+;;; obtaining a copy of this software and associated documentation
+;;; files (the "Software"), to deal in the Software without
+;;; restriction, including without limitation the rights to use, copy,
+;;; modify, merge, publish, distribute, sublicense, and/or sell copies
+;;; of the Software, and to permit persons to whom the Software is
+;;; furnished to do so, subject to the following conditions:
+;;;
+;;; The above copyright notice and this permission notice shall be
+;;; included in all copies or substantial portions of the Software.
+;;;
+;;; THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+;;; EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+;;; MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+;;; NONINFRINGEMENT.  IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+;;; HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+;;; WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+;;; OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+;;; DEALINGS IN THE SOFTWARE.
+
+(in-package #:babel-encodings)
+
+(define-character-encoding :cp1252
+    "A 8-bit, fixed-width character encoding used by Windows for Western
+    European languages."
+  :aliases '(:windows-1252)
+  :literal-char-code-limit 256)
+
+(define-constant +cp1252-to-unicode+
+    #(#x20ac    nil #x201a #x0192 #x201e #x2026 #x2020 #x2021
+      #x02c6 #x2030 #x0160 #x2039 #x0152    nil #x017d    nil
+         nil #x2018 #x2019 #x201c #x201d #x2022 #x2013 #x2014
+      #x02dc #x2122 #x0161 #x203a #x0153    nil #x017e #x0178)
+  :test #'equalp)
+
+(define-unibyte-decoder :cp1252 (octet)
+  (if (and (>= octet #x80) (<= octet #x9f))
+      (svref +cp1252-to-unicode+
+             (the ub8 (- octet #x80)))
+      octet))
+
+(define-constant +unicode-0152-017e-cp1252+
+    #(#x8c #x9c #x00 #x00 #x00 #x00 #x00 #x00
+      #x00 #x00 #x00 #x00 #x00 #x00 #x8a #x9a
+      #x00 #x00 #x00 #x00 #x00 #x00 #x00 #x00
+      #x00 #x00 #x00 #x00 #x00 #x00 #x00 #x00
+      #x00 #x00 #x00 #x00 #x00 #x00 #x9f #x00
+      #x00 #x00 #x00 #x8e #x9e)
+  :test #'equalp)
+
+(define-constant +unicode-2013-203a-cp1252+
+    #(#x96 #x97 #x00 #x00 #x00 #x91 #x92 #x82
+      #x00 #x93 #x94 #x84 #x00 #x86 #x87 #x95
+      #x00 #x00 #x00 #x85 #x00 #x00 #x00 #x00
+      #x00 #x00 #x00 #x00 #x00 #x89 #x00 #x00
+      #x00 #x00 #x00 #x00 #x00 #x00 #x8b #x9b)
+  :test #'equalp)
+
+(define-unibyte-encoder :cp1252 (code)
+  (cond
+    ((or (< code #x80)
+         (and (> code #xa0) (<= code #xff)))
+     code)
+    ((and (>= code #x0152) (<= code #x017e))
+     (svref +unicode-0152-017e-cp1252+
+            (the ub8 (- code #x0152))))
+    ((= code #x0192) #x83)
+    ((= code #x02c6) #x88)
+    ((= code #x02dc) #x89)
+    ((and (>= code #x2013) (<= code #x203a))
+     (svref +unicode-2013-203a-cp1252+
+            (the ub8 (- code #x2013))))
+    ((= code #x20ac) #x80)
+    ((= code #x2122) #x99)
+    (t (handle-error))))

I hope you will find it useful.

Regards,

octets-to-string with UTF8 and Byte Order Marker

Rob Blackwell — 2011-04-06T10:07:23

Hi,

 

I have some byte arrays which are UTF8 and some which are UTF8 with byte
order markers.

 

I can convert these arrays to strings using

 


 

and

 


 

respectively, but I'm currently having to figure out whether there is a
BOM, like this

 


#(239 187 191)

 

If I use (babel:octets-to-string foo) on a byte array with BOM markers,
then my SBCL Lisp image dies.

 

Is there a better way to ask Babel to discover the correct encoding by
looking for Byte Order Marks? Ideally I'd like one function call that
worked with any array and figured out which encoding was being used
automatically and works whether or not a BOM is present?

 

Sorry if I'm missing something obvious, I'm a Babel newbie .. Any
guidance or code samples gratefully received.

 

Thanks,

 

Rob.

 

_______________________________________________
babel-devel mailing list
babel-devel< at >common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/babel-devel

Unicode encodings with explicit endianness

Dmitry Ignatiev — 2010-10-28T17:50:07

Hello again, Luís

Please, can you hurry up a bit with applying my unicode-related patch?
I'm currently in process of developing a library, a binding to ms windows
api:
http://github.com/Lovesan/doors
And the reason of me being worried about that patch is because my library is
unable to correctly operate on windows unicode strings without UTF-16LE
support in babel.
_______________________________________________
babel-devel mailing list
babel-devel< at >common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/babel-devel

UTF-16/32 encodings with explicit endianness

Dmitry Ignatiev — 2010-10-03T08:44:32

_______________________________________________
babel-devel mailing list
babel-devel< at >common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/babel-devel

question about #\Nul char and Unicode

Vsevolod Dyomkin — 2010-08-04T14:07:18

_______________________________________________
babel-devel mailing list
babel-devel< at >common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/babel-devel

Version 0.3.1 is requried

Andrey Moskvitin — 2009-12-26T11:29:52

_______________________________________________
babel-devel mailing list
babel-devel< at >common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/babel-devel

STREAM-LINE-COLUMN for VECTOR-OUTPUT-STREAMs

Ariel Badichi — 2009-12-04T12:52:01

Hello,

The vector output stream implementation is missing an applicable
method for STREAM-LINE-COLUMN, which is required for character output
support.  Patch follows.

Ariel

diff -rN -u old-babel/src/streams.lisp new-babel/src/streams.lisp
--- old-babel/src/streams.lisp2009-11-29 11:31:24.000000000 +0200
+++ new-babel/src/streams.lisp2009-11-29 11:31:25.000000000 +0200
< at >< at > -323,6 +323,11 < at >< at >
      (extend-vector-output-stream-buffer sequence stream :start start :end end)))
   sequence)
 
+(defmethod stream-line-column ((stream vector-output-stream))
+  "Dummy line-column method that always returns NIL, needed for
+character output streams."
+  nil)
+
 (defmethod stream-file-position ((stream vector-stream))
   "Simply returns the index into the underlying vector."
   (declare (optimize speed))

Unibyte decoders/encoders return wrong values

Ariel Badichi — 2009-11-29T01:15:02

Hello,

Unibyte decoders/encoders return (- START END) rather than (- END START).
Patch follows.

Ariel

diff -rN -u old-babel/src/encodings.lisp new-babel/src/encodings.lisp
--- old-babel/src/encodings.lisp2009-11-29 03:01:45.000000000 +0200
+++ new-babel/src/encodings.lisp2009-11-29 03:01:45.000000000 +0200
< at >< at > -375,7 +375,7 < at >< at >
                      (declare (type code-point ,',code))
                      (block ,',encoding ,< at >',body)))
                  ,',dest ,',di)
-                finally (return (the fixnum (- ,',d-start ,',di))))))))
+                finally (return (the fixnum (- ,',di ,',d-start))))))))
 
 ;;; The decoder version of the above macro.
 (defmacro define-unibyte-decoder (encoding (octet) &body body)
< at >< at > -400,7 +400,7 < at >< at >
                      (declare (type ub8 ,',octet))
                      (block ,',encoding ,< at >',body)))
                  ,',dest ,',di)
-                finally (return (the fixnum (- ,',d-start ,',di))))))))
+                finally (return (the fixnum (- ,',di ,',d-start))))))))
 
 ;;;; Error Conditions
 ;;;

japanese encoding support.

SANO Masatoshi — 2009-11-16T15:50:10

Hi.

The attached file is japanese encoding (euc-jp and cp932) support for babel.
Could you merge it?

XCVB patch for Babel

Faré — 2009-10-09T18:46:38

Here is the patch I use to build Babel with XCVB (assuming alexandria
and trivial-features are similarly patched).

Is it OK to include this patch upstream?

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
[...] there is what I call  the "roundtrip fallacy": it is a mistake to use,
as journalists and some economists do, statistics without logic but the reverse
does not hold:  It is not a mistake to use logic without statistics.
        -- N. C. Taleb, Fooled by Randomness, 2004.
_______________________________________________
babel-devel mailing list
babel-devel< at >common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/babel-devel