gmane.comp.python.stackless

Re: channels and StopIteration

Kristján Valur Jónsson — 2012-05-25T11:02:53

Hi richard.
I think you misunderstand.
The raised exception is indeed the same instance on the other side.  send_throw does not, in fact, build on send_exception but is an entirely new api.
It creates a bomb object using the passed in exception information.
The python api is designed to be similar to the generator.throw() function:

The manual says this:

generator.throw(type[, value[, traceback]])
Raises an exception of type type at the point where generator was paused, and returns the next value yielded by the generator function. If the generator exits without yielding another value, a StopIteration exception is raised. If the generator function does not catch the passed-in exception, or raises a different exception, then that exception propagates to the caller.

But the implementation is different, accepting the same kind of arguments as a ´raise' exception.
The implementation for channel_send_throw borrows the argument processing from generator_throw().

I've submitted a defect to bugs.python.org about

Re: channels and StopIteration

Richard Tew — 2012-05-25T03:54:28

On Sat, Apr 14, 2012 at 12:00 AM, Kristján Valur Jónsson
<kristjan-hPzAiOG7rkJWk0Htik3J/w< at >public.gmane.org> wrote:

The change in general looks good except for two aspects that make me antsy.

The first is the API of the throw implementation.

http://hg.python.org/stackless/rev/d4b2e291fb48#l2.82

The way this accepts an existing exception instance as an argument..
You can make something that looks like the thrown instance on the
other side, sure, but it isn't the same as the thrown one.  I do not
find it consistent and I would want the raised tasklet to be the one
that was thrown, if I were throwing a specific instance of one.  If
this isn't going to happen, then I'd want to know about it and get an
exception should I try.  I'd insert a pithy phrase here about being
sold some object and being told it is another, but I don't remember
any :-)

The second is that this builds on the existing Stackless tasklet
exception raising mechanism.  And that mechanism is very limited
(specify a class and simple argument

Re: stackless bug fix

Richard Tew — 2012-05-25T03:19:52

On Tue, May 22, 2012 at 2:54 AM, Kristján Valur Jónsson
<kristjan-hPzAiOG7rkJWk0Htik3J/w< at >public.gmane.org> wrote:

Looks good.

Cheers,
Richard.

Re: Tasklet cleanup?

Richard Tew — 2012-05-25T03:11:48

Hi Sylvain,

Kristjan having clarified that TaskletExit is definitely off the table
for now, I'd like to make the response I should have when you first
posted.  Fix your circular references and __del__ should be fine.

What I tend to do is use weak references, where applicable, in order
to ensure that nothing keeps a tasklet alive except where I explicitly
want.  There is nothing internal to Stackless which will keep a
tasklet alive without your implicit direction.  If my tasklets are
kept alive and I detect it, then I use gc.get_referrers(tasklet) to
track down why.

Cheers,
Richard.

Re: Is the Stackless website down?

Richard Tew — 2012-05-24T22:45:53

Ah, I hope I didn't put you on the spot quoting Christian.  Note that this
is the first time in months that we have had this problem - and despite our
phone problems we can make do.  Had I emailed Christian directly earlier,
chances are we could have had minimal downtime and a quick reboot.

Looking forward to the end of phone and zone whenever it comes :-)

Cheers,
Richard.
_______________________________________________
Stackless mailing list
Stackless-KA+417sA3NeakBO8gow8eQ< at >public.gmane.org
http://www.stackless.com/mailman/listinfo/stackless

Re: Tasklet cleanup?

Kristján Valur Jónsson — 2012-05-23T19:14:28

Yes, i´m sure we need to revisit this.

Re: Tasklet cleanup?

Sylvain Prat — 2012-05-23T14:41:25

Hello,

1. The solution I proposed in my previous email doesn't work since the
last dying tasklet would never get a chance to be cleaned up.

2. I finally sorted my tasklet cleanup problem out by breaking the
reference cycle. I didn't succeed to break the cycle by using a
weakref on the cycle: some elements were garbage collected too early.
I finally added a level of indirection, namely a "current" variable
which holds the currently executing wrapper, which works similarly to
stackless.current. Then my function use the "current" variable and is
not a bound method anymore. Reference cycle broken, and now I can use
__del__ to kill my tasklets/wrappers and get a similar behavior than
greenlets for cleanup. I hope I am clear ;)

3. I still consider that an TaskletExit should be raised for cleanup,
but at least, I have found a workaround ;)

Cheers,
Sylvain


On Mon, May 21, 2012 at 4:24 PM, Sylvain Prat <sylvain.prat-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

stackless bug fix

Kristján Valur Jónsson — 2012-05-21T14:54:42

I've just fixed a bug where the data sent over a channel could get lost, being replaced with a "None".
This could happen if channel action, tasklet deletion and stack spilling all happened at once :)
See
http://blog.ccpgames.com/kristjan/
for details.

K
_______________________________________________
Stackless mailing list
Stackless-KA+417sA3NeakBO8gow8eQ< at >public.gmane.org
http://www.stackless.com/mailman/listinfo/stackless

Re: Tasklet cleanup?

Sylvain Prat — 2012-05-21T14:24:02

Thanks for the clarification Kristján! So, we should not expect a
TaskletExit to be raised ;)

I fully understand the problem with __del__ and circular references,
which is also documented there:
http://docs.python.org/reference/datamodel.html#object.__del__

The problem is, when I try to cleanup resources by myself, I also get
myself into the __del__ & circular references trap, and I can't find a
workaround. If I try to replace a hard reference by a weakref in the
circular references loop, I also get some errors (I've not
investigated them yet, maybe they are avoidable). I suspect the only
way to get cleanup work is to do something in stackless itself.

Also, what do you mean by "it were safer to put such tasklets into
gc.garbage"? How can we do the cleanup then? Do you mean we have to
poll gc.garbage (maybe in a cleaning tasklet), identify tasklets that
have to be destroyed, and kill them manually, so that we can control
the moment when the tasklet will be destroyed?

About the problem of "switching" to t

Re: Tasklet cleanup?

Kristján Valur Jónsson — 2012-05-21T11:21:26

Well.
It is due to this code here, in taskletojbect:

static int
tasklet_traverse(PyTaskletObject *t, visitproc visit, void *arg)
{
    PyFrameObject *f;
    PyThreadState *ts = PyThreadState_GET();

    /* tasklets that need to be switched to for the kill, can't be collected.
     * Only trivial decrefs are allowed during GC collect
     */
    if (tasklet_has_c_stack(t))
        PyObject_GC_Collectable((PyObject *)t, visit, arg, 0);

What can be done during garbage collection has always been a bit restricted.  Tranditionally, gcmodule disallows all objects with __del__ methods and puts them in gc.garbage.  I have recently discovered, by asking on python-dev, that this is not due to a fundamental fragility of the garbage collection process, but rather the fact that it is likely that __del__ methods invoked during garbage collection will cause exceptions, since the members of the cycle would refer to each other.  See this discussion:  
http://grokbase.com/p/python/python-dev/124hc36dtk/issue-9141-finalizers-

Re: Is the Stackless website down?

Alain Poirier — 2012-05-21T09:46:38

Hi all, hi Richard, Hi Christian,

Le 19 mai 2012 à 00:37, Richard Tew a écrit :


Yes, we've already set a Trac instance and migrated most of the contents.
We still have some works to complete before to propose a new version.


Now knowing the poor state of the actual site, we will hurry to finish the
migration. I hope to be able to show a new version to Christian this week.


Best regards,
Alain

Re: Tasklet cleanup?

Sylvain Prat — 2012-05-20T20:02:40

Hello,

The simple program I'm trying to get working is attached to this
email. As you can see from the program execution :
- the destructor is not called
- the cleanup is not performed
- the __del__ method make the wrapper, the tasklet and its frame
uncollectable. If I remove the destructor, everything is collectable,
but I still don't get the cleanup performed.

I expected a TaskletExit to be raised but I thought something was bad
in my code. But seeing your examples, it's not just my code ;)

Also, I got no success with tasklet.kill: as I suspected, this method
is used to kill an existing tasklet in user code, and is not called
"automatically" by stackless when a tasklet is garbage collected.

Even if we miss a TaskletExit in stackless, are there workarounds to
make my small program work?

Cheers,
Sylvain

Re: Tasklet cleanup?

Gil Colgate — 2012-05-20T04:09:38

I may be mistaken, but I believe the *kill* method is called on the tasklet
during garbage collection.
You can clean stuff up in that, but you are not allowed to raise any
uncaught exceptions.

On Sat, May 19, 2012 at 3:29 PM, Sylvain Prat <sylvain.prat-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>wrote:

_______________________________________________
Stackless mailing list
Stackless-KA+417sA3NeakBO8gow8eQ< at >public.gmane.org
http://www.stackless.com/mailman/listinfo/stackless

Re: Tasklet cleanup?

Richard Tew — 2012-05-20T00:05:57

Greenlet is derived from Stackless, so similarly TaskletExit should be
raised on a tasklet that is being garbage collected.


I've attached two example scripts where a tasklet dies, but does not
get the TaskletExit raised on it.  This is something Kristjan Valur
has been working on, but there have been upsides and downsides to the
different approaches and as I understand it the ideal solution to
tasklet destruction is yet to be found.

Kristjan, why is TaskletExit not being raised?  Any ideas?

Cheers,
Richard.
_______________________________________________
Stackless mailing list
Stackless-KA+417sA3NeakBO8gow8eQ< at >public.gmane.org
http://www.stackless.com/mailman/listinfo/stackless

Tasklet cleanup?

Sylvain Prat — 2012-05-19T22:29:19

Hello,

I'm wondering how tasklets can clean themselves up when they are
destroyed due to garbage collection (i.e. when they are not in the
runnables and not referenced by any object anymore). Greenlet solves
this problem by raising a GreenletExit exception in the greenlet's run
function when the greenlet is about to die due to garbage collection.
However, in stackless, it seems that no TaskletExit exception is
raised when the tasklet is about to die, so we can't simply use a
try/finally in the tasklet's callable to clean up resources.

I tried to wrap my tasklet in a parent object which has the same
lifespan as my tasklet and has a __del__ function for cleaning up, but
I keep having problems with circular references (the wrapper/parent
object also provides the callable of the tasklet, i.e. a bound method)
that make the tasklet/parent object uncollectable (circular references
: wrapper --> tasklet --> stackless machinery? --> callable stack
frame (bound method of wrapper) --> wrapper). Same problem when tryi

Re: Is the Stackless website down?

Richard Tew — 2012-05-18T22:37:41

Alright, emailed Christian and got this reply:

"""
yes, this is the old broken plone site. It slowly eats all memory and then
crashes after a couple of weeks.  The Nagare people told me at
PyCon that they are re-building it with trac, but I didn't hear back since
then. Need to contact them now. I forgot about that, again.
"""

The site has been rebooted and now works again.  I'll add a link to my
mirror to the official download page, and hopefully google cache or
some such malarky can point the way when next plone crashes.

Cheers,
Richard.

Re: Is the Stackless website down?

Richard Tew — 2012-05-18T22:07:42

No problem.  Transfer finished by the way.

Cheers,
Richard.

Re: Is the Stackless website down?

Fernando Miranda — 2012-05-18T22:03:34

Thanks a lot.

Re: Is the Stackless website down?

Richard Tew — 2012-05-18T21:54:10

No official mirror.  But I've ftp'd the binaries directory here:

http://disinterest.org/resource/stackless/binaries/

Note that it is still transferring.

I also tried restarting apache on the stackless host.. but that didn't
do the trick.  The web site will still have to wait for Christian to
fix it.

Cheers,
Richard.

Re: Is the Stackless website down?

Richard Tew — 2012-05-18T21:52:06

Christian is probably not reading this list.  No-one else knows how to
manage the web site.  There's no real trouble except for the
inconvenience of the web site not being available.

Richard.

Re: Is the Stackless website down?

Gil Colgate — 2012-05-18T21:47:41

Is there some trouble going on in the stackless world?

On Fri, May 18, 2012 at 11:55 AM, Fernando Miranda <fcmiranda-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>wrote:

_______________________________________________
Stackless mailing list
Stackless-KA+417sA3NeakBO8gow8eQ< at >public.gmane.org
http://www.stackless.com/mailman/listinfo/stackless