gmane.comp.lib.libevent.user

Re: questions about bufferevent callbacks in multithreaded code

Sashan Govender — 2013-05-16T14:09:30


Thanks. I'll give 2.1.3-alpha a go. I'll also try and strip away more of my
code till I have a reproducible case for libevent or I find the problem in
my code. I've tried enabling the multithreaded debugging
(evthread_enable_lock_debugging) but that doesn't trigger any assertions.

Re: questions about bufferevent callbacks in multithreaded code

Nick Mathewson — 2013-05-15T14:37:47

Right.


Right, so long as there isn't some kind of a deadlock scenario where
the two threads are acquiring locks in different order.

If you *are* hitting a mysterious bufferevent-related deadlock, though
,you might want to check out Libevent 2.1.3-alpha: it has a fix for a
longstanding tricky deadlock in bufferevents related to how
event_del() handles an event that is currently running its callback.

Re: questions about bufferevent callbacks in multithreaded code

Sashan Govender — 2013-05-15T10:24:59

So in theory the following scenarios should not deadlock:

This all happens in a single thread:
1) a bufferevent read callback is triggered (lock acquired)
2) the user supplied callback function reads the data and calls
bufferevent_write (lock acquired - ok because using a recursive mutex)

and neither should this where 2 threads are involved:
1) a bufferevent read callback is triggered
2) the user supplied callback function does some stuff (callback ends ->
lock released by libevent)
3) bufferevent_write is called from another thread


On Thu, May 2, 2013 at 2:12 AM, Nick Mathewson <nickm< at >freehaven.net> wrote:

Re: Is evthread_use_pthreads required for isolated evdns event_bases?

Nick Mathewson — 2013-05-09T15:19:47

 [...]

I'm indifferent, so long as user code that works today continues to work.

Re: Is evthread_use_pthreads required for isolated evdns event_bases?

Joseph Spadavecchia — 2013-05-09T09:49:36


Anytime.  This looks like a fun task.  I should be able to start work on it next week.

[…]

Sounds good.  Shall I make the PRNG not shared by default, or do you want it the other way around?

Many thanks.
*Joe

***********************************************************************
To unsubscribe, send an e-mail to majordomo< at >freehaven.net with
unsubscribe libevent-users    in the body.

libevent kqueue not working properly on fd returned from zmq_getsockopt()

Pritesh Acharya — 2013-05-08T17:27:11

This problem is also related to zmq. I'm writing a service in C programming
using libevent and zmq. Msg is pushed from python code to C service using
PUSH-PULL pattern.

fd received from zmq socket:

void *receiver = zmq_socket (base.zmq_ctx, ZMQ_PULL);
zmq_connect (receiver, "tcp://localhost:5557");int fd=0;size_t fd_len
= sizeof(fd);
zmq_getsockopt (receiver, ZMQ_FD, &fd, &fd_len);

Using Libevent, event registered with fd for persistent read

struct event *read_data_on_zmq =event_new(base.evbase, fd, EV_READ |
EV_PERSIST , read_data_on_zmq_cb,receiver);
event_add(read_data_on_zmq,NULL);
event_base_dispatch(base.evbase);

On the callback method I'm doing a non-blocking receive

void read_data_on_zmq_cb(evutil_socket_t fd, short what, void *arg){
    char *msg = calloc(1024,sizeof(char));
    int size = zmq_recv (receiver, msg, 255, ZMQ_DONTWAIT);
    if (size != -1)
    {
        puts ("is size is not -1");
        printf("msg = %s\n",msg);
    }

    zmq_getsockopt (receiver, ZMQ_EVENTS, &fd, &fd_size);

Re: Is evthread_use_pthreads required for isolated evdns event_bases?

Nick Mathewson — 2013-05-05T23:03:57

 [...]

A patch would definitely be a good help.  Thank you!  The most
important thing to me here would be that the abstraction needs to be
good; the evdns_base should be holding some kind of "struct
evutil_secure_random_state", so that it's easier to swap in a better
backend than arc4random later.

Github pull requests probably work best these days for me.

best wishes,

Re: Is evthread_use_pthreads required for isolated evdns event_bases?

Joseph Spadavecchia — 2013-05-03T17:12:50

Hi Nick.

On Fri, 3 May 2013 11:47:13 -0400, Nick Mathewson <nickm< at >freehaven.net>
wrote:
[...]

Oh, I didn't know there was a shared PRNG in evdns.  I thought evdns_base
behaves like event_base, i.e., that they shared nothing by default.

[...]

Thanks.  That would work for me.

[...]

My only thought is perhaps the PRNG should be per evdns_base by default.

Oh, and if you could use some help, then I'd be happy to submit a patch.

Many thanks.
*Joe

***********************************************************************
To unsubscribe, send an e-mail to majordomo< at >freehaven.net with
unsubscribe libevent-users    in the body.

Re: Is evthread_use_pthreads required for isolated evdns event_bases?

Nick Mathewson — 2013-05-03T15:47:13

 [...]

Under the current architecture, you do indeed need to initialize
threading for any resource to be shared across threads, which *does*
include the PRNG.  Unfortunately, evdns does not currently have an
option like EVENT_BASE_FLAG_NOLOCK, so once threading is initialized,
every evdns_base is going to have a lock that it doesn't need.

I'd be open to work to change this, including by:
  * Letting the PRNG be thread-specific.
  * Adding an option to make the PRNG be evdns_base-specific
  * Adding a flag like EVENT_BASE_FLAG_NOLOCK for evdns_base initialization.

Other thoughts?

yrs,

Is evthread_use_pthreads required for isolated evdns event_bases?

Joseph Spadavecchia — 2013-05-03T13:41:56

Hi there,

I'm developing a multithreaded proxy server (running on Linux 2.6.32;
libevent-2.0.20-stable), where each event_base is isolated to a single
thread.

As I understand, by isolating the event_bases per thread and not calling
evthread_use_pthreads(), I can avoid locking overheads.  AFAIK,
evthread_use_pthreads() only needs to be called when multiple threads share
an event_base.  If this is incorrect, then please disregard this email.
Otherwise, read on... :)

On one particular site, after about two weaks of heavy load (e.g. 500
req/sec), the evdns.c:transaction_id_pick() gets stuck in an infinite loop
(all threads using evdns get stuck and requests cannot be served).

Via a gdb debug session, I found that transaction_id_pick()'s call to
evutil_secure_rng_get_bytes() always returns the same trans_id.  I could
not get any further due to compiler optimizations, so I rebuilt libevent
without optimizations and allowed it to run for another two weeks on the
customers site.  Again, evdns.c:transaction_id_pi

[possibly OT] Updated giant inventory of headers available on various OSes

Zack Weinberg — 2013-05-02T03:34:07

A while ago I posted a call for help (only to autoconf< at >) taking inventories
of which C headers are provided by which OSes.  It rapidly became clear
that my methodology was far too crude, and I've spent much of the past six
weeks coding up something much more sophisticated.  The project is now at
the point where I feel comfortable sharing it with a wider audience.
Please see http://hacks.owlfolio.org/header-survey/ for the results so far,
and https://github.com/zackw/header-survey/ for code, raw data, and
instructions for contributing inventories for OSes not yet in the big
table.  (Big table only known to work properly in Firefox, say sorry -
patches for the bad stylin' in Webkit most welcome.  I've exhausted my
ability to sling CSS.)

If you sent me an inventory already but your OS is not in the big table, I
need you to send it again using the current `scansys.py`.  To those of you
who told me that it didn't work with your version of Python, take heart -
it is now tested to work with 2.0.  Yes, 2.0.

zw

ANN: Libevent 2.1.3-alpha is released!

Nick Mathewson — 2013-05-02T03:10:53

Announcing Libevent 2.1.3-alpha
===============================

Libevent 2.1.3-alpha is released!

This is the third release in the Libevent 2.1 series. As the "alpha"
should tell you, this is not a stable release: you should not run it
unless you're willing to debug together.  If you want a stable version
of Libevent, stick to the 2.0 series for now.

The package is available from the website, at http://libevent.org/

There are GPG signatures there too; you should probably verify them,
to make sure you get the software you think you are getting.

What's new in Libevent 2.1.3-alpha
==================================

This release has a bunch of stuff, including:

   * An experimental "finalization" feature to try to solve the
     deadlock in multithreaded bufferevent use.
   * Numerous bug and portability fixes.

For explanations of what's changed in 2.1 so far, see "What's New In
Libevent 2.1", available as "whatsnew-2.1.txt" in the source distribution,
or online at https://raw.github.com/libevent/libeven

Re: questions about bufferevent callbacks in multithreaded code

Nick Mathewson — 2013-05-02T01:12:25

Ordinarily, yes. You can override this with the
BEV_OPT_UNLOCK_CALLBACKS option, as described in
http://www.wangafu.net/~nickm/libevent-book/Ref6_bufferevent.html


Yes.

Yrs,

questions about bufferevent callbacks in multithreaded code

Sashan Govender — 2013-05-01T16:07:29

Hi

I need to clarify my understanding of how bufferevent objects work with
mulithreaded code. If a bufferevent triggers a callback (read or write)
does it hold a lock until that callback completes? Do the locks use a
recursive mutex?

Thanks

Re: Bufferevent filter with greedy sender

Nick Mathewson — 2013-04-28T01:45:05

I think you want to look into watermarks: it's a way to tell Libevent
"Don't let this buffer get more full than X" or "don't bother me
unless this buffer has at least X bytes".

I'm too distracted to post good code for using watermarks together
with filters tonight, but the idea should be pretty straightforward
from the documentation and the reference manual.  (If somebody _can_
write a good example, possibly nice and terse for inclusion in the
manual, that would rock.)  Let us know if it still doesn't make sense
or doesn't work for you though.

peace,

Re: Bufferevent filter with greedy sender

kgardenia42 — 2013-04-27T18:45:02

I should add: the low level read callback does eventually get called
but only after hundreds of calls to the filter's input callback.  What
I'm asking for is a way to call the read callback sooner so the dst
buffer doesn't get bloated.  How can I control this behavior?

On Sat, Apr 27, 2013 at 6:52 PM, kgardenia42 <kgardenia42< at >googlemail.com> wrote:
***********************************************************************
To unsubscribe, send an e-mail to majordomo< at >freehaven.net with
unsubscribe libevent-users    in the body.

Bufferevent filter with greedy sender

kgardenia42 — 2013-04-27T17:52:24

Hi,

I have a bufferevent_filter to parse some protocol data from the "src"
evbuffer and unwrap it into the "dst" evbuffer.

This all works fine.  The problem I have, however, is that when there
is a greedy sender sending a lot of data the input_filter just keeps
getting called hundreds or thousands of times, massively bloating the
dst buffer.

What I would *like* to happen is for the lower level read callback (on
the bev which is wrapped by the filter) to be called so that dst gets
drained and I don't get the bloat.

I've read the docs and I can't seem to grock this part of it and
couldn't find a relevant example.  Can someone please advise how to
accomplish this?

I see that the filter callback has a "enum bufferevent_flush_mode
state" parameter.  Is this relevant here?

Thanks,
***********************************************************************
To unsubscribe, send an e-mail to majordomo< at >freehaven.net with
unsubscribe libevent-users    in the body.

Re: Simple question about multithreading SSL bufferevents.

Nick Mathewson — 2013-04-25T21:26:54

Cool!  In that case, the best way to test would be to either take the
current git master, then merge in the branch "21_deadlock_fix" from
https://github.com/nmathewson/libevent/ .

Alternatively, you could wait for 2.1.3-alpha, which I hope will be
out pretty soon.

best wishes,

Re: Simple question about multithreading SSL bufferevents.

John — 2013-04-25T21:23:38

Nick,

I have made my code capable of running multiple sets of single-threaded SSL bufferevent+event_base (i.e. works with current libevent 2.0) as well as able to run fully multithreaded (breaks with current libevent 2.0) I will be happy to test the changes in my real softwarewhich would probably be a better stress test than the simple test software I posted before.

Thanks,
-John


________________________________
From: Nick Mathewson <nickm< at >freehaven.net>
To: libevent-users< at >freehaven.net; John <jjchnk-libevent< at >yahoo.com> 
Sent: Thursday, April 25, 2013 11:36 AM
Subject: Re: [Libevent-users] Simple question about multithreading SSL bufferevents.


On Tue, Mar 26, 2013 at 6:32 PM, John <jjchnk-libevent< at >yahoo.com> wrote:

FWIW, I can confirm that my current "21_deadlock_fix" branch makes
your example code run as expected.  See the thread "RFC: strategy for
deadlock avoidance and safe finalization in Libevent 2.1" from earlier
this month for more information on that.  I hope to clean it up in the
next few d

Re: Simple question about multithreading SSL bufferevents.

Nick Mathewson — 2013-04-25T18:36:02

FWIW, I can confirm that my current "21_deadlock_fix" branch makes
your example code run as expected.  See the thread "RFC: strategy for
deadlock avoidance and safe finalization in Libevent 2.1" from earlier
this month for more information on that.  I hope to clean it up in the
next few days and merge it soon. Ideally, I would like to get the next
Libevent 2.1.x-alpha released over this weekend.

best wishes,

Re: Could not use event_base_foreach_event

Nick Mathewson — 2013-04-25T13:54:33

On Thu, Apr 25, 2013 at 1:24 AM, Pritesh Acharya
<priteshacharya< at >gmail.com> wrote:

No, but it's only in Libevent 2.1, not in Libevent 2.0.  It sounds
like maybe you're building with headers from Libevent 2.1, but linking
against libevent 2.0.