gmane.lisp.drakma.devel

Votre Convention collective

Michel Deloncle — 2013-04-11T19:16:55

Problem with SBCL, Drakma, and quicklisp

Patrick May — 2013-03-23T14:03:38

I'm trying to get Drakma working with SBCL.  I've loaded it with quicklisp but when I try to connect to a website in Slime, I get this error:

The function DRAKMA::WITH-CHARACTER-STREAM-SEMANTICS is undefined.
   [Condition of type UNDEFINED-FUNCTION]

Any idea what could be causing this?  I've attached the backtrace in case that helps.

Thanks,

Patrick

Backtrace:
  0: ("undefined function")
  1: (HTTP-REQUEST #<PURI:URI http://ichart.yahoo.com/table.csv?s=IBM&a=0&b=01&c=2012&d=11&e=31&f=2012&g=d&ignore=.csv>)
  2: (GET-YAHOO-SYMBOL-DATA "IBM" "2012-01-01" "2012-12-31")
  3: (SB-INT:SIMPLE-EVAL-IN-LEXENV (GET-YAHOO-SYMBOL-DATA "IBM" "2012-01-01" "2012-12-31") #<NULL-LEXENV>)
  4: (EVAL (GET-YAHOO-SYMBOL-DATA "IBM" "2012-01-01" "2012-12-31"))
  5: ((LAMBDA () :IN SWANK:EVAL-AND-GRAB-OUTPUT))
  6: (SWANK::CALL-WITH-RETRY-RESTART "Retry SLIME evaluation request." #<CLOSURE (LAMBDA # :IN SWANK:EVAL-AND-GRAB-OUTPUT) {13BBF7E5}>)
  7: (SWANK::CALL-WITH-BUFFER-SYNTAX NIL #<CLOSURE (LAMBDA # :IN SWANK:EVAL-AND-GR

Error while trying to POST some JSON data

Peter Kuchta — 2013-03-21T21:41:35

Hi everybody,

I'm trying to POST some JSON data to a web service:

        (ql:quickload :st-json)
        (ql:quickload :cl-json)
        (ql:quickload :drakma)

        (defvar *rc* (merge-pathnames (user-homedir-pathname) ".apirc"))

        (defvar *user*
          (with-open-file (s *rc*)
            (st-json:read-json s)))

        (defvar api-url (st-json:getjso "url" *user*))
        (defvar api-key (st-json:getjso "key" *user*))
        (defvar api-email (st-json:getjso "email" *user*))

        (setf drakma:*header-stream* *standard-output*)

        (defvar *req* '(("dataset" . "tigge")
                  ("step"    . "24")
                  ("date"    . "20071001")
                  ("time"    . "00")
                  ("origin"  . "all")))

        (format t "json:~S~%" (json:encode-json-to-string *req*))

        (defun retrieve (api request)
          (let* ((cookie-jar (make-instance 'drakma:cookie-jar))
                 (extra-headers (list (cons "From" api-email)

Drakma documentation overhaul

Hans Hübner — 2012-12-09T21:04:32

Hi,

while fixing some bugs in Drakma recently, I was notified that the
documentation and the docstrings were not matching well anymore.  I
have decided to overhaul the documentation, and in the process of
that, I also made some source changes.

There will be more changes soon, so you may experience instabilities
when you're using the master branch from github.

Comments regarding the documentation would be very welcome, I have put
a version up for review at http://netzhansa.com/drakma.html

Thanks,
Hans

error while parsing set cookie containing httponly &secure

Simon Koch — 2012-12-09T17:15:08

When parsing this Set-Cookie line:

Set-Cookie: shssl=4058628; path=/; secure; HttpOnly

The resulting Cookie is:

#<COOKIE shssl=4058628; path=/; domain=www.base.de>

Which misses both features: 'HttpOnly' and 'secure'.

I traced the bug down to 'parse-set-cooie' which returns
(("Set-Cookie: shssl" "4058628" (("path" . "/") ("secure; HttpOnly"))))

instead of (("Set-Cookie: shssl" "4058628" (("path" . "/") ("secure")
("HttpOnly"))))

As far as I understood the code the problem is caused by
'read-name-value-pairs' of chunga.

I am using chunga 1.1.1 and drakma 1.2.9

Is this a already known bug/problem?

charset errors question

Jeff Cunningham — 2012-09-24T17:01:47

I've been running into some trouble using drakma to retrieve pages from 
certain commercial websites. It is very likely the HTML they are 
generating is broken one way or another. But the problem still remains 
as to how one can retrieve their pages using drakma.

For example, if you try this simple case:

    (http-request "http://www.walmart.com")

It will display the following:

WARNING: Problems determining charset (falling back to binary):
Corrupted Content-Type header:
Read character #\;, but expected #\=.

And the returned body is binary-encoded ascii. This can be converted to 
real ascii, of course, but it is inconvenient to say the least.

Often the problem is that their metatag for the charset is simply wrong. 
Sometimes I can figure out what it is and supply this information, like 
this:

    (http-request "http://www.walmart.com" :external-format-in :UTF-8)

and it will solve he problem. But this particular example does not lend 
itself to this, at least using the following charsets:

  :UTF-8

about character set of cookie name

lookwong — 2012-06-28T07:22:42

Here is a complete discussion on the character set of cookie name  (for
python development):
http://bugs.python.org/issue2193

I think *drakma *should follow python's work.

In file "util.lisp" of *drakma*, function *split-set-cookie-string*  splits
cookie string using function* is-valid-cookiename-char* which comes from *
chunga*.
But function* is-valid-cookiename-char *just follow *RFC2616* which  rules
that chars of cookie name are legal when they are in* token set *( see
bellow)*.
*

  token          = 1*<any CHAR except CTLs or separators>
  separators     = "(" | ")" | "<" | ">" | "< at >"
                      | "," | ";" | ":" | "\" | <">
                      | "/" | "[" | "]" | "?" | "="
                      | "{" | "}" | SP | HT

*
*That is not practical!*

*In python, letters, digits and "!#$%&'*+-.^_`|~:"*
<http://hg.python.org/cpython/rev/8cae3ee7f691#l1.15>  *are legal in Cookie
name.
_______________________________________________
drakma-devel mailing list
drakma-devel< at >common-lisp.net
http://list

Changing content-type seems to have no effect

Michael Minerva — 2012-03-15T17:53:16

I have been trying to get my lisp application to interface and upload a file to my drupal 7 website using Drakma.  

So far I have been able to to connect to my drupal site and authenticate and store the session cookie (thank you Ryan Davis, I just found your response the other day).  Here is the code I use to do that:

(with-output-to-string (stream)
  (setf drakma:*header-stream* stream)
(let ((cookie-jar (make-instance 'drakma:cookie-jar)))
  (drakma:http-request
   "http://localhost:8888/?q=rest/user/login"
    :method :post
    :cookie-jar cookie-jar
    :parameters 
    '(("username" . "login")
      ("password" . "password"))
    )
  (drakma:http-request
   "http://localhost:8888/?q=rest/file"
    :method :post
    :cookie-jar cookie-jar
    :content-length t
    :parameters '(("file1" . #p"/Users/Mike/hello.png"));("file1" #p"/Users/Mike/Desktop/mario-gif.gif" :content-type "image/gif" :filename "mario-gif.gif"))
    :content-type "application/x-www-form-urlencoded" 
    ;:parameters 
    )))


This

Unwanted url-encoding of GET parameters.

William Halliburton — 2012-03-10T00:08:22

Hello folks,

I'm trying to use drakma to fetch urls that contain utf8 characters but
HTTP-REQUEST automatically url encodes any non latin-1 ascii characters.

On my cursory reading of the RFCs, this seems conforming behavor, but in
this case it is definitely unwanted.

For example, the following url

  http://translate.google.com/translate_tts?tl=ru&q=вы

if entered directly into the browser correctly returns the text-to-speech
audo file but

when attempting to use HTTP-REQUEST, the url is being url encoded into

  http://translate.google.com/translate_tts?tl=ru&q=%D0%B2%D1%8B

of which google does not url-decode and fails to return the correct data.

So, for this case, the url-encoding is unwanted.

I am willing to submit patch an additional argument into HTTP-REQUEST to
disallow the encoding.

Thoughts?

Thank you,
William
_______________________________________________
drakma-devel mailing list
drakma-devel< at >common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel

drakma and non-ASCII content

Robert Brown — 2012-03-09T15:24:44

A co-worker of mine had some problems today using Drakma to POST a STRING
containing non-ASCII characters encoded as UTF-8.  He was doing something
equivalent to:

(http-request "http://zappa.com/favicon.ico"
              :method :post
              :content (concatenate 'string
                                    "hello" (string #\white_square) "world")
              :external-format-out :utf-8)

which ends up setting the Content-Length header value to 11, which is the
LENGTH in characters of the string being sent.

The documentation says that "if content is a sequence, Drakma will use LENGTH
to determine its length and will use the result for the 'Content-Length' header
sent to the server," so Drakma is working as documented.

Also, I think I understand why this behavior is the default.  You don't want to
scan a content string in order to determine what its length will be in octets
once it has been encoded.  My co-worker could have used a vector of type
(unsigned-byte 8) to hold his UTF-8 encoded data.

H

http redirect following url-encodes location

Willi Nesensohn — 2012-02-27T19:20:29

I'm using drakma to GET data over http. It turns out that when following 
redirects, drakma uses the location in the redirection header, which it 
url-encodes. Usually this might not be a problem, but when the location 
redirected to is already url-encoded this method fails.

Is this intended behaviour?

I'm not sure what the clean solution to this would be, as decoding the 
location before using it just doesn't feel right. For the lack of a 
better solution that's what i'm doing right now though.

Any pointers in the right direction would be greatly appreciated.

Best regards,
Wilfried Nesensohn

A simple Cookie and viewing the actual Post

Michael Minerva — 2012-02-09T20:41:17

I just started using drakma today and so far it seems really great but I had a couple quesitons.

First, is it possible to send a simpler for example i want my http-post to have something like this:

Cookie: SESSdc0685ed01f285dab628a3700259e6bc=rCfmWLUeugRQNAQTs14yox_DWqZyGE2fYUS8W24j4co

I tried making a drakma cookie and adding it to a cookie-jar but none of that seemed to work.

Also, is there a way to print the actual post I am sending so I can see what might be wrong with it?

Thanks a lot,

--Mike

Drakma and client certificates?

Rob Blackwell — 2012-01-14T12:15:11

Hi,

I'm trying to use client certificates with Drakma to call an existing web based API.

I have the latest code from https://github.com/edicl/drakma installed in my quicklisp local-projects directory. I'm using SBCL on OS X and Linux.

The client certificate is a .pfx file, and I've converted it to .pem format using

openssl pkcs12 -in robblackwellmanage.pfx -out robblackwellmanage.pfx.pem

I have verfied that this works as a server certifcate, using hunchentoot, thus:

(hunchentoot:start (make-instance 'hunchentoot:ssl-acceptor :ssl-privatekey-file "/Users/reb/certs/robblackwellmanage.pfx.pem" :ssl-certificate-file "/Users/reb/certs/robblackwellmanage.pfx.pem" :ssl-privatekey-password "password" :port 4343))

When I try to use this as a client certificate:

 (drakma:http-request resource
 :certificate "/Users/reb/certs/robblackwellmanage.pfx.pem"
 :key "/Users/reb/certs/robblackwellmanage.pfx.pem"
 :certificate-password "password"
 :method :get
 :content ""
 :content-type "application/xm

cl-oauth and drakma interoperability issue

mohegan-skunkworks — 2011-12-29T18:01:45

I just filled an issue on github...

The text is below...
Hi,
I'm using cl-oauth to authenticate with twitter (it's in the cl-twitter package, also on github).

I received a complaint from a quicklisp user that he couldn't authenticate with twitter.

After looking into this a bit I realized that drakma is encoding the uri on get requests. Now, cl-oauth passes in an encoded uri so this uri gets encoded twice. This was never a problem for me because I was using v 1.2.3 of drakma where this didn't happen.

Here's the code diff (plus my proposed change :)
========================== v 1.2.3 ==================================
(when (and (not parameters-used-p)
parameters)
(setf (uri-query uri)
;; append parameters to existing query of URI
(format nil "~< at >[~A~]~:*~:[~;&~]~A"
(uri-query uri)
(alist-to-url-encoded-string parameters external-format-out))))

===========================current github/v1.2.4=============
(when-let (all-get-parameters
(append (dissect-query (uri-query uri))
(and (not parameters-used-p) par

HTTP Put and form-data

Ben Holm — 2011-11-13T21:54:44

Hi all. I started using drakma the other day to do some client work with
reviewboard, and I had a little trouble. Reviewboard requires http puts
using form-data for updates (
http://www.reviewboard.org/docs/manual/1.6/webapi/2.0/overview/). Drakma
explicitly disallows this.

I modified request.lisp to let my client do what I needed. Since this is my
first time writing a webservice client, I'm not sure if reviewboard or
drakma is wrong, or if I misunderstood something. Below is the diff of what
I did.

Any thoughts?

Thanks,
-Ben

Index: request.lisp
===================================================================
--- request.lisp (revision 4683)
+++ request.lisp (working copy)
< at >< at > -410,7 +410,7 < at >< at >
     (parameter-error "Don't know how to handle scheme ~S." (uri-scheme
uri)))
   (when (and close keep-alive)
     (parameter-error "CLOSE and KEEP-ALIVE must not be both true."))
-  (when (and form-data (not (eq method :post)))
+  (when (and form-data (and (not (eq method :put)) (not (eq method
:post))))
     (

read-status-line is seeing the HTTP request line!?

Daniel Weinreb — 2011-03-17T15:30:21

Has anybody seen a behavior in which read-status-line
signals a syntax-error, and it turns out that the value
of "line" looks like an HTTP request line rather
than an HTTP status (response) line?  The following
is the stack trace.

 From my old C++ days (may they never return), I
conjectured that some string buffer is being reused
or something like that.; We are running on CCL
(formerly openmcl), and nothing like that is
going on at the CCL level.; I don't know much
about flexi-streams or gray-streams. Anyway
that's just a rather wild guess. Thanks!

<READ-STATUS-LINE at pc +669
     Arguments: (STREAM &amp;OPTIONAL DRAKMA::LOG-STREAM)
       STREAM: <FLEXI-STREAMS:FLEXI-IO-STREAM #x30200938EB3D>
       DRAKMA::LOG-STREAM: NIL
     Local bindings:
       CHUNGA:CURRENT-ERROR-MESSAGE "While reading status line:""
       DRAKMA::LINE: ""POST /stat/ping HTTP/1.1""
       DRAKMA::FIRST-SPACE-POS: 4
       DRAKMA::SECOND-SPACE-POS: 15

Thanks!

- Dan

http-request error

André Schütz — 2011-02-28T15:50:06

Hello,

I loaded the drakma system successfully into my clisp environment and got the following error when I tried to make a http-request:

(http-request "http://lisp.org/")

Fehler: NODELAY in SOCKET-CONNECT is unsupported.

Any ideas?

Thanks in advance
Andre Schütz

Failure to load under SBCL on MS Windows 7 (64-bit)

Duncan Bayne — 2011-01-21T04:41:37

Hi All,

I'm using Drakma via Quicklisp on SBCL.  This works fine under Ubuntu
10.10, but when I attempt to quickload Drakma on MS Windows Ultimate
64-bit, I get 'Error opening shared object "libssl32.dll"'.

Here's a REPL session that demonstrates the problem:

=====
* (load (merge-pathnames "quicklisp/setup.lisp" (user-homedir-pathname)))

T
* (ql:quickload '("drakma"))
To load "drakma":
  Load 1 ASDF system:
    drakma
; Loading "drakma"
................
debugger invoked on a CFFI:LOAD-FOREIGN-LIBRARY-ERROR:
  Unable to load foreign library (LIBSSL).
  Error opening shared object "libssl32.dll":
  126.

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RETRY          ] Try loading the foreign library again.
  1: [USE-VALUE      ] Use another library instead.
  2: [TRY-RECOMPILING] Recompile reload and try loading it again
  3: [RETRY          ] Retry loading component ("cl+ssl" "reload").
  4: [ACCEPT         ] Continue,

[Patch]: better file-names when sendingmultipart/form-data from streams.

Stas Boukarev — 2010-12-26T14:01:45

The attached patch does two things:
* :filename option takes precedence of automatically determined names
 (it was the case previously only for pathnames, and not for streams or functions).
* if stream is a file-stream, which means it's a pathname designator,
use file-namestring to get the filename.

Patch for disabling SSL in drakma

Marko Kocić — 2010-10-06T09:47:52

Hi all,

I'm sending a quick patch to disable CL+SSL dependency, as well as
https support in drakma when :drakma-no-ssl feature is present.
It works in a similar way as :hunchentoot-no-ssl for hunchentoot.
It also added quick info about it in "Download and installation"
section of the documentation.

The motivation is to be able to use drakma on windows where working
libssl32 is not easy to be found.

Regards,
Marko Kocic
_______________________________________________
drakma-devel mailing list
drakma-devel< at >common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel

drakma patch: Don't try to funcall keywords

Faré — 2010-09-28T15:41:08

This bit me when updating Drakma. Note that another function in same
file uses the same colloquialism as this one: testing if a symbol is
fbound before trying to funcall it, otherwise just print it - which
works great for keywords.

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
Don't forget your daily prayer to Baah-kup,
the God of data storage and recovery!
_______________________________________________
drakma-devel mailing list
drakma-devel< at >common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel