gmane.network.beanstalk.general

Fwd: beankstalkd questions I couldn't find the answers to

Keith Rarick — 2013-06-19T06:45:50

Here's a recent email exchange, in case anyone has similar questions.

---------- Forwarded message ----------
From: Colleen Ross <cross-FNGGT++BwowAvxtiuMwx3w< at >public.gmane.org>
Date: Mon, Jun 17, 2013 at 8:12 AM
Subject: Re: beankstalkd questions I couldn't find the answers to
To: Keith Rarick <kr-1tNhcOpH3NA< at >public.gmane.org>

That's fine. Thanks for the answers!

On Friday, June 14, 2013, Keith Rarick <kr-1tNhcOpH3NA< at >public.gmane.org> wrote:

High CPU and much slower performance in 1.9

2013-06-17T20:54:24

I am prototyping a queueing service for my company that utilizes a couple 
thousand tubes.  I have a test case where I run 3000 tubes with 1200 jobs 
each in them.  Those tubes are serviced by a distributor that takes work 
and funnels it to a couple of much larger work queues where the real work 
happens.  I have 3 clients with 10 threads each processing these larger 
tubes and sending results into yet another tube.

I use delay in the many feeder tubes.  I do not use delay in the large work 
queues.  All priorities are the same.

I was having some sporadic crashes due to connection issues in 1.8 so I 
upgraded to 1.9.  Since then the crashing has gone away but my test case 
takes 3x longer to run and CPU is maxed out the entire time.  With 1.8 CPU 
would spike during the initial load of the 3000 queues X 1200 jobs but then 
would settle down to very little during the time the queues were being 
serviced.

In terms of real numbers, my system used to process 600k jobs through the 
entire system in about 5 mi

Re: Beanstalk 2.0

Keith Rarick — 2013-06-12T21:47:48

On Wed, Jun 12, 2013 at 3:17 AM, OmarShariffDontLikeIt
<omarshariffdontlikeit-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

The 1.x series won't make *incompatible* changes, but
other changes are fine.

For our purposes, a change is compatible if we can
expect existing clients to continue to work as designed.
Some examples of protocol changes that have
happened since 1.0:

- add a unique id and hostname to stats output
- new command: kick-job
- allow underscore in tube names


Most of the outstanding features in the issue tracker
can comfortably be implemented in a compatible way,
including the two you listed. I'd love to see them go in.
Most of these features require some design work as
well as implementation. I'm happy to collaborate on
design issues, even though I haven't had time lately
to work on implementation for new beanstalkd features.

Most of my beanstalkd time in recent months has
been spent fixing important bugs as they arise. That,
along with some janitorial work in the code, continues
to be

Beanstalk 2.0

OmarShariffDontLikeIt — 2013-06-12T10:17:49

It's my understanding that the current 1.x version of Beanstalk will
not implement changes to the beanstalk protocol. I'm sure everyone has
their pet features they would love adding to beanstalk (my personal
ones are Atomic Move between Tubes (https://github.com/kr/beanstalkd/
issues/170) and Clear Tube Command (https://github.com/kr/beanstalkd/
issues/25)).

My question is what is the current roadmap for Beanstalk 2.0? I'd like
to know if these or any other features/issues will be addressed and
when.

P.S. As a developer myself I know how cringe-worthy the question is :)

Re: Best approach to handle job dependencies

BlackKitten — 2013-06-12T07:46:48

... 

Hi Jesus, 
          thanks for the ideas, the fact that checking and re-inserting the 
job is quite wasteful is a very good point.
I will make some tests with many tubes (so each tube will become a kind of 
"level") and benchmark it a bit.

many thanks :)

Re: Best approach to handle job dependencies

Jesús Gabriel y Galán — 2013-06-11T18:03:47

In this case I don't see how this worker is not "a big do-all worker",
unless the worker would first check if it should be handling this task
at this point and reinserting the job if not, which is highly wasteful
in my opinion, cause tasks could be enqueued several times until the
appropriate worker happens to take it.


If you want to have independent workers, each doing its own job, I
would have one tube per worker, and having, as you even said, the
output of a worker to be the input of the next one. You could have in
the task payload the info of the sequence, to that each worker can
know which tube to reinsert the task.

So, in your example, Task1 will be queued in tube "A" and task 2 and 3
in tube "C". Worker A will process task 1, check which is the next one
in sequence for this task, and enqueue the task in tube "B". Worker B
would then process the task and insert it in tube "C", and so on. In
the meantime, worker C would have processed task 2, and then task 3,
inserting it in tube "A".

For the seque

Best approach to handle job dependencies

BlackKitten — 2013-06-11T08:23:27

Hi all,
    I am new to beanstalk and just moving first steps: everything look very 
nice up to now :)
I would like to ask you an advice to manage job dependencies: say for 
example that my application "tasks" can be split into several workers that 
must be run sequentially, for example:

Task id=1: WorkerA --> WorkerB --> WorkerC --> Done.
Taks id=2: WorkerC --> Done.
Taks id=3: WorkerC --> WorkerA --> Done.

etc. The idea is that the output of a Worker is the input of the next one, 
but the real game is that the order is not fixed.
I guess I can build a big "do-all" worker, but this will not scale as 
nicely...

My first idea is to have the N worker create the N+1 job in a common tube, 
passing the application-specific info as payload, so only one tube.

What do you think? Any ideas?  
Thanks!

Re: stuck delayed job in 1.8

James Bardin — 2013-06-10T02:17:56


On Saturday, June 8, 2013 2:47:11 AM UTC-4, Keith Rarick wrote:

Hmm, maybe that was it. Didn't think about that since this is only running 
with one tube.... except for "default". Dammit.

Re: stuck delayed job in 1.8

Keith Rarick — 2013-06-08T06:47:11

Was the job in a different tube? It's possible your
peek and kick commands were operating on one
tube, and the job was in another. The global stats
command gives totals for all tubes.

Re: stuck delayed job in 1.8

James Bardin — 2013-06-07T13:58:56

An upgrade to 1.9 didn't change anything.
However the job did get processed last night.

I does concern me though, that I couldn't peek, or kick the delayed job at 
all.

-jim


On Thursday, June 6, 2013 11:40:34 AM UTC-4, James Bardin wrote:

stuck delayed job in 1.8

James Bardin — 2013-06-06T15:40:34

I've got a running beanstalk showing 1 delayed job, but I can't access it.

1
None
0

Restarting the beanstalkd doesn't change anything.

Jaume Sabater — 2013-06-02T12:20:15

http://elitekitchensandvanities.com.au/junnt/ahocputqyq.fdvpmjfr

beanstalk console available as a Perl PSGI application

Karen Etheridge — 2013-05-27T00:35:58

I've written a Perl wrapper for the PHP Beanstalk Console application,
which makes the server mountable as a standard PSGI/Plack endpoint.

https://metacpan.org/release/Plack-App-BeanstalkConsole
https://github.com/karenetheridge/Plack-App-BeanstalkConsole

thanks again to Petr Trofimov and Sergey Lysenko for the PHP app!

Karen Etheridge
ether-a09SyBuiYrA< at >public.gmane.org

Beantop: A command line tool

Alejandro Siri — 2013-05-23T10:28:27

Hi All,

I needed a way to keep an eye on the status of several tubes on my 
Beanstalkd on production, so I coded up https://github.com/alexsiri7/beantop. 
I didn't know if there was anything else similar so I started it from 
scratch. 

Feel free to use it, fork it, or send requests, I will still be updating it 
when I can.

If you think it is worth it, we could put it up on the website somewhere so 
people can have fast access to it.

Cheers,
Alex

Re: Can I use beanstalkd to queue up tasks that can be executed at a specific time during the day?

Jesús Gabriel y Galán — 2013-05-22T09:55:09

Thanks for your answer.
What I am doing is managing many paid subscription services. Users
subscribe to the service, and every week I have to bill them. If the
billing fails, I have a retry policy within the week, that depends on
each service. I don't mind to-the-second accuracy, but I wouldn't like
to skip any billing try for several hours, so it's fine.


In my proof of concept, I never delete the jobs, I only release with
delay. This means that even if the client fails, the job is always in
the queue, correct?
Anyway, what I wanted to understand was if there was a way to inspect
the jobs that are in the queue, to check that every user has at least
one job in the queue. If there isn't, the only way is to check that a
job has executed for the user in the last week, or to insert jobs for
all users every once in a while, but this might be overkill.

Thanks,

Jesus.

Re: Can I use beanstalkd to queue up tasks that can be executed at a specific time during the day?

Keith Rarick — 2013-05-22T05:17:22

On Tue, May 21, 2013 at 12:37 AM, Jesús Gabriel y Galán
<jgabrielygalan-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

It's hard to say whether this is good without knowing some
more details about what you're doing. But if you've already
tried it and it works well, then I'd say it's a good approach.


That will probably be more reliable in the long run. Beanstalkd
isn't a database. You might consider a periodic cron job that
scans the database looking at timestamps, and generates
beanstalkd jobs to do whatever work is necessary. But again,
evaluating these approaches is hard to do in the abstract.

Re: Can I use beanstalkd to queue up tasks that can be executed at a specific time during the day?

Jesús Gabriel y Galán — 2013-05-21T07:37:31

Forgive me for jumping in this thread, but I had a similar use case in
mind that could be solved by Beanstalkd and I would like to ask your
opinion.

I have to perform a task for each user (of which I have thousands) in
a weekly basis, but at a different time for each user, based on each
user's subscription time for example. Also each task can fail and
should be retried with a policy that can be dynamic. So I made a proof
of concept in which when a user subscribed I insert a job in
beanstalkd with a week delay. Then n workers will listen for jobs, get
one, make a try and if it fails return it to the queue with a
calculated delay (5 minutes or 1 day, for example).

Do you think this fits correctly? In my case I'm not really concerned
with being accurate to the second in processing the job, I just need
that it's run approximately one week after the previous one.

There's also a concern in that I would like to be able to check every
once in a while that all users have a corresponding job in the queue,
that I'm

Re: Can I use beanstalkd to queue up tasks that can be executed at a specific time during the day?

Chad Kouse — 2013-05-21T01:01:34

what he said.

Re: Can I use beanstalkd to queue up tasks that can be executed at a specific time during the day?

Keith Rarick — 2013-05-21T00:38:54

On Mon, May 20, 2013 at 5:05 AM, Abishek.R Srikaanth
<abishekrsrikaanth-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

Yes, but I wouldn't recommend it.

You can approximate it by subtracting the current time
from the time you want and adding a delay for that duration.
But there's no guarantee the job will start on time; you must
make sure there are workers working at the right time and
that no other jobs are ahead in line keeping them busy.

The general meaning of beanstalkd's delay is to wait *at
least* the specified amount of time.


If you need cron, use cron.

Can I use beanstalkd to queue up tasks that can be executed at a specific time during the day?

Abishek.R Srikaanth — 2013-05-20T12:05:15

Can I use beanstalkd to queue up tasks that can be executed at a specific 
time during the day?
Would I be able to replace beanstalkd in performing jobs that a cron can 
perform?

Thanks
Abishek R Srikaanth

Re: How many queues can beanstalkd handle?

Ben. — 2013-05-14T17:45:04

Thank you for your reply.

This helps me a lot.

Regards
Ben

Am Dienstag, 14. Mai 2013 00:27:27 UTC+2 schrieb Keith Rarick: