gmane.comp.lang.pike.user

Re: multiple return types?

Henrik Grubbström — 2012-05-17T15:24:27


It's a known limitation in the parser that the Pike compiler uses.
You can work around it by using the old-style syntax:

   object(.Frame)|mapping start_frame;

or by swapping the order of the types:

   mapping|.Frame start_frame;

--
Henrik Grubbströmgrubba< at >roxen.com
Roxen Internet Software AB

multiple return types?

Lance Dillon — 2012-05-17T14:27:10

I have a class method that returns either an object (.Frame) or a mapping.  I have the function defined as such:

.Frame|mapping read_frame();

but when I try to define a variable:

.Frame|mapping start_frame;

I get the following error:

Connection.pike:45:syntax error, unexpected TOK_MAPPING_ID

However, if I change it to object instead of .Frame:

object|mapping start_frame;

It works fine.

Is this an error somewhere?  Or is that a syntax error on my part?

Re: Crypto.MD5

Chris Angelico — 2012-05-15T22:36:05

Nope. Might be faster to use the short version, is definitely briefer.
They're guaranteed equivalent.

ChrisA

Re: Crypto.MD5

Lance Dillon — 2012-05-15T22:29:54

Actually, I just did:

String.string2hex(Crypto.MD5.hash(str));

Is there a difference between what is below and what I did?  Crypto.MD5()->update(str)->digest() vi Crypto.MD5.hash(str)?

Thanks




>

Re: Crypto.MD5

David Emanuel da Costa Santiago — 2012-05-15T21:59:52

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hello!

String.string2hex(Crypto.MD5()->update("INPUT")->digest());

Best regards,
David Santiago


Em Tue, 15 May 2012 08:12:15 -0700 (PDT)
Lance Dillon <riffraff169< at >yahoo.com> escreveu:


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iQIcBAEBAgAGBQJPstHhAAoJEAiFd8wEHg6FjxcP/ibOZFbTICn+HMHvaYlzY5RZ
ok4vIHUYE/D8KJzTgKMYpcd1FetD6nDdVWqfBv8u4YtVb5823tE77o7k480CfLCK
QiiMV42vjve/xoHEM/bbg3Ou0PdlEWfxg+cO9j34N7NQkzDp7KamYXjWjZ8jEXOE
6AyTKhNk45P/GgJs3FForqf8fXvLhm5dX2i48Eaf59J+h3qn3gXE+a4gTKmPNgtQ
8W5mh8vUH9uPfly1z2TgmP4Ek/KuwMWVSNkoKaUA/6lQwwX6Qd8LzDdaGcdmWVI5
NVnEcSp9K9M8gQOtcHXtvrqZPQDmiTWFwqHIBb+kJ3HlTo2QK7WisOp/lowlqX3v
vVon4+9gFWb1gmmpcxTmooS8TluQPQZzD4t5NW82Et8s5k/ogv/EY8Z/8SRufOzi
qgpqlpzKtcWiuLZkP6rVOp7w4PU2boh21zoREQwae5ckCJQAIMdh0Y37iFcVEti+
my+PQjWP9lgf6f/k20KpEMyViCHsPq4uQKdW21mOpXrmVuQfkDqm4v8XGEv7qEyw
27Gyk/Xh04q5/pa/24QoMxVEon3z0tIVV0SqOjQt25hLjpOCqVVBZqZy0v0VZnM1
toeT8Nu5/QVR2TpRKfAPGkTGpZ3cGne+rItyxX/e

Re: Crypto.MD5

Henrik Grubbström — 2012-05-15T15:29:57


If you don't use Safari (it will crash), you can try the (experimental)
manual at

http://pike.lysator.liu.se/generated/manual/modref/ex/predef_3A_3A/Crypto/MD5.html

which does list the inherited symbols (when javascript is enabled).


As you've noticed, the digest is returned as a binary string.


There's no need for the array_sscanf(). Just cast the string to an 
array(int).


There are multiple ways, the most common is to use String.string2hex().

A more exotic one is

   Gmp.mpz(str, 256)->digits(16);

--
Henrik Grubbströmgrubba< at >roxen.com
Roxen Internet Software AB

Crypto.MD5

Lance Dillon — 2012-05-15T15:12:15

I want to generate a hash exactly as md5sum would.  What would be the best way of doing that?  The docs don't seem to be that helpful.

http://pike.ida.liu.se/generated/manual/modref/ex/predef_3A_3A/Crypto/MD5.html

doesn't really say how to use it, other than it inherits from MD5_Info (Nettle.MD5_Info) and Hash (.Hash).

Nettle.MD5_Info just says it is an internal mixin class.
.Hash just says it is an abstract class, with only `()() listed, but it inherits Nettle.HashInfo
Nettle.HashInfo has block_size(), digest_size(), hash(), and name().

md5sum on a certain file game me:
2fcb2fbe1ff9e1dd6cfa10429879008

Crypto.MD5.hash() gave me "/\313/\276\37\371\341\335l\372\20B\230y\0\213"

I'm assuming there is some sort of transformation after Crypto.MD5.hash() to get the same result as md5sum, I just don't know what that is.

Actually, with a little experimenting, and looking at rfc1321 (http://www.ietf.org/rfc/rfc1321.txt), page 20, I came up with the following code.

string md5sum(string str) {
    string ha

Re: newbie questions - "finally"

Henrik Grubbström — 2012-05-04T15:13:32


The last comment is not quite true, since the lock variable may get reused 
if there are further variable declarations after the block.


True.

--
Henrik Grubbströmgrubba< at >roxen.com
Roxen Internet Software AB

Re: SQLite and pike strings in bindings

Henrik Grubbström — 2012-05-04T15:11:10


If I remember correctly, the Oracle glue uses strings wrapped in multisets 
to indicate BLOBS in bindings. This should be doable in the SQLite case as 
well.

--
Henrik Grubbströmgrubba< at >roxen.com
Roxen Internet Software AB

Re: newbie questions - "finally"

Martin Stjernholm — 2012-04-21T17:28:52


That ought to make it a lot trickier to write C modules for Python. I
imagine there are plenty of C functions that implement their own delay
systems in more or less ad hoc ways, then. In pike the core provides
that service instead.


Yes, but in the case of variables with block scope there's also the
matter of zeroing the storage immediately when leaving the block,
something I believe pike also can delay, to be able to combine pop
instructions. But it still doesn't seem like a big win to me.


Basically, no. Just like Python, Perl and most other script languages, I
believe. I've thought a bit on how to change that; there's a
mast/multi-cpu branch in git, but it only has a long document describing
what needs to be done (see http://github.com/pikelang/Pike/blob/mast/
multi-cpu/multi-cpu.txt). It's not easy.

(Btw, sorry for linking to github instead of our own
pike-librarian.lysator.liu.se, but the text file viewer there is broken,
at least in my FF.)

Re: newbie questions - "finally"

Chris Angelico — 2012-04-21T16:22:14

Python simply acknowledges that arbitrary Python code could be run any
time you DECREF an object. But since you're saving them off to a
linked list, it's basically the same thing only delayed a little; the
efficiency cost is practically zero, unlike a stop-the-world GC being
run that often.


Immediately, bypassing the linked list. It's not fundamentally different.


Sure, but it's impossible to switch between Pike threads in the middle
of, say, an increment operation.

(Is Pike able to make use of multi-CPU systems (or multi-core CPUs) at
all? This would suggest no.)


Yeah... I was just reading up on that sort of thing this week, with
things like the http://en.wikipedia.org/wiki/Cyrix_coma_bug that can
be used to wedge a CPU by preventing task switches. But you're right,
it's still preemptive. It would be more correct to say that it's the
granularity of the context switches that gives the measure of
convenience.

ChrisA

Re: newbie questions - "finally"

Martin Stjernholm — 2012-04-21T15:59:55


Doesn't Python have destructors? In that case there's a lot more going
on than just freeing it.

When an object loses its last ref, pike doesn't call the destructor
right away. That's because we don't know the execution environment there
- it might not be safe to throw exceptions, there may be very little
stack, some svalue pointers may be temporarily invalid, etc. Since
destructors are pike code we've no idea how much code gets run before
destroy() returns, and what it might do.

So we have to move the object to a linked list, and then go through that
list at a better time, i.e. when we're back in the interpreter. It must
be a list, because we can't count on that only one object gets freed
between two such interpreter points.

I figure the problem ought to be pretty much the same in Python - they
just choose to destroy objects more frequently.


Well, you'd get quite a lot of markers then. It's designed so you can't
write any sizable amount of pike code without getting yield points in
it, regardless wheth

Re: newbie questions - "finally"

Chris Angelico — 2012-04-21T14:30:09

Python doesn't run the GC every time something gets decref'd to zero,
it just frees that one object.


Ah. Yeah, on second thoughts, probably would add a pile of mess to the code.


Yes, but it's still under control, unlike what I normally think of as
preemptive multitasking (where a context switch can happen between any
two machine instructions). Switching happens at places that you can
mark in your high level code.


Heh. One of these days I need to deploy a Roxen, just to see what it's
like to work with. I wonder if I can convince my boss to shift off
Apache+PHP... probably not, the amount of code-rewrite for the amount
of benefit doesn't look "sellable". But some day I'm going to set me
up a Roxen at home.

ChrisA

Re: newbie questions - "finally"

Martin Stjernholm — 2012-04-21T14:20:59


It's more efficient to leave the garbage around for a little while
longer and free more things in one go. But how big the difference would
be I can't say. Probably not very much.


What do you mean? It doesn't exactly get simpler by running it more
often.


The expense would normally be setting up and tearing down a catch
context. It's not particularly expensive, but it adds a small bit of
overhead of course. But more importantly it's too blunt - I don't want
to catch any error and just ignore it, so I'd have to check the error
message and rethrow etc, which is very clumsy.


I'd classify the current approach as preemptive since threads are
switched without any explicit calls on the pike level.


Well, it's just such a quirky implicit behavior. Not at all pretty. But
considering how much code Roxen has which depends on it, I'd say it
isn't very likely it'll change substantially.

Re: newbie questions - "finally"

Chris Angelico — 2012-04-21T13:28:07

It'd be interesting to evaluate the pros and cons of these two
techniques. Is there a simplification possible to the GC this way?


Interesting. Hasn't ever been a problem to me, since most threaded
applications that I create are extremely I/O-bound (for instance, a
MUD with one thread per socket connection, allowing the thread to
block for input whenever it wants to), but worth knowing all the same.


How expensive are exceptions? Would it be safer to simply attempt the
activation, and catch the error if misc->cachekey is 0?

Although, cooperative multithreading is a lot easier than perfectly
preemptive. Might be simpler to document it and make it official.

ChrisA

Re: newbie questions - "finally"

Martin Stjernholm — 2012-04-21T12:58:29


Yes, it'd be reasonable to assume that a variable is freed immediately
when it goes out of scope, but it's not so (currently, at least). That
can be surprising with mutex locks, for instance:

     {
       Thread.MutexKey lock = mutex->lock();
       // Got the lock here.
     }
     // Still got the lock here, even though the lock variable is
     // inaccessible.

So it's a good idea to always destruct mutex keys explicitly when
leaving a locked region, unless it extends to the function return. Same
goes for closing files.


Correction: The count should be 3 instead of 2, because there'll be a
ref on the stack as well, for the function argument when _refs is
called.


Originally it was. I guess the reason is that the refcounts can exhibit
various internal behaviors which might change, e.g. optimizations might
cause things to get more or less refs in various situations.


There are no official rules - proper pike code should always assume a
thread switch can take place at any moment. In practice, thread

Re: SQLite and pike strings in bindings

Marc Dirix — 2012-04-20T22:04:49

I think none in this case. Because the column is defined as TEXT, the 
column has TEXT affinity, meaning that the
column can store NULL, TEXT or BLOB. However if given "unknown" type (in 
SQL usually quoted with ''), or integer etc, they are automatically 
converted to the affined type: TEXT. The conversion takes place before 
the comparison.

A cast will work:
"SELECT * from abc WHERE bla=CAST('foo' AS BLOB);"

/Marc

Re: SQLite and pike strings in bindings

Stephen R. van den Berg — 2012-04-20T13:48:10

What would be the (if any) proper syntax/quoting to make the above SELECT
work with the way the data is in the current DB?

Re: SQLite and pike strings in bindings

Marc Dirix — 2012-04-20T12:04:42

PostgreSQL datatypes are determined during table creation, and fixed 
thereafter.
So if you define a column as TEXT, or VARCHAR(n) you can only insert 
these values having these types (or a typeconversion takes place).
I don't now the ins and outs of the PSQL module, but it probably lets 
the database decide wether an string is data (BYTEA) or not

In SQLite this is not fixed. If you define a column to be TEXT you can 
just as well insert other datatypes, it simple changes
the datatype for the record. The SQLite glue code in the module defines 
whether a string is data or not, using sqlite3_bind_blob and 
sqlit3_bind_text functions.

The main problem I find is the mixed behaviour with inserted string. If 
my string is 1-byte, the SQLite module inserts it as "BLOB"
in the SQLite database and therefor I can't select it using a prepared 
SELECT, (e.g. "SELECT * from abc WHERE bla='foo'"), whereas for 2 or 
4-byte strings this works.

Using bindings for both SELECT and INSERT works as expected.

/Marc

Re: SQLite and pike strings in bindings

Stephen R. van den Berg — 2012-04-20T11:46:18

I'm wondering, why don't I run into this problem in the pgsql driver?
Does the SQLite driver do something fundamentally different with respect
to handling data than PostgreSQL?

Re: newbie questions - "finally"

Chris Angelico — 2012-04-20T02:46:15

That's what I meant, poor choice of words on my part.


Ah, interesting. This is unlike, for instance, Python, where the
instant the last reference is removed, the object is deallocated. On
that model, the object would be destroyed at the close brace of the
while loop - the point at which that last ref disappears.


Ooh that's a cool technique. I'll bear that in mind! I didn't take
particular notice of that function as the docs say that it's meant
mainly for debugging Pike itself.


Hmm, I've never looked into when Pike switches between threads, mainly
because I don't use threads in situations where that can matter. What
are the rules governing that? Is there documentation someplace?

ChrisA