gmane.comp.lang.ruby.god

Monitoring Unicorn Workers with God

Nitesh Goel — 2012-05-09T19:16:35

There seems to be no easy to way monitor unicorn workers with god. Github 
has a hack, where they use a separate thread to monitor the workers, but 
the post is two and a half years old 
(https://github.com/blog/519-unicorn-god).

Have things changed since?

Re: Different restart commands based on the restart condition

Nitesh Goel — 2012-05-09T01:59:19

Hi Nuno
Thanks. What I meant was this:

Currently I can specify a start, stop, and restart command in god.
I want to specify multiple different restart commands.

E.g.
Let's say I am monitoring the thin web server. If my box is out of memory, 
I want to restart with 5 servers, but if my box is out of cpu, I want to 
start with 3 servers (don't ask why .. contrived example).

So ..

w.restart1 = thin restart --servers 5
w.restart2 = thin restart --servers 3

How do I inject the restart command to run in the restart_if block?

On Monday, May 7, 2012 6:23:09 PM UTC-4, Valente wrote:

Re: Different restart commands based on the restart condition

Nuno Valente — 2012-05-07T22:23:09

Hi Nitesh. Sure you can.
You can find examples in the god homepage but i'll leave you one. You just
need to wrap your conditions in a block.

w.restart_if do |restart|

        #Custom conditon
        restart.condition(:log_watcher) do |c|
          c.monitor_file w.log.to_s
          c.add_regexp(/.+OutOfMemory.+/)
        end
        #Built-in god conditon
        restart.condition(:memory_usage) do |c|
              c.above = 100.megabytes
              c.times = [3, 5] #3 out of 5 measurements
        end
        #Built-in god conditon
        restart.condition(:cpu_usage) do |c|
          c.above = 98.percent
          c.times = 3 # 3 consecutive measurements
        end

end

Happy monitoring.

Best regards,
Nuno Valente.

On Mon, May 7, 2012 at 9:47 AM, Nitesh Goel <me-wfq69V0LymhGBRGhe+f61g< at >public.gmane.org> wrote:

Different restart commands based on the restart condition

Nitesh Goel — 2012-05-07T08:47:03

Hey Guys
I am monitoring a process where my restart command is based on the restart 
condition failed. E.g.

If memory usage too high:

w.restart = "command 1"

if cpu usage too high:

w.restart = "command 2"

Is there a way to configure a watch as such?

Thanks.

God-rb doesn't auto-daemonize unicorn well?

Guang-Nan Cheng — 2012-05-06T09:07:16

God stores a wrong pid for auto-demonizing unicorn. I can get things 
working by set the pid file on both unicorn and god. But do I have to do it 
this way? Why god doesn't auto-daemonize unicorn well?

god overwrites ".kilobytes" in Rails

Stefan Adamov — 2012-04-25T23:28:58

Setup:

Gemfile
gem 'god'

rails c
irb: warn: can't alias exit from irb_exit
=> 5120

work fine if I follow https://github.com/mojombo/god/issues/32

Gemfile
gem 'god', :require => false

rails c
=> 5242880

Re: how to start the process

Nuno Valente — 2012-04-17T14:10:24

No. God is just a process. You need to create an init.d script to start at
boot time or some other method to boot it and load your recipes.

Best regards,
Nuno Valente.

On Tue, Apr 17, 2012 at 2:42 PM, Gitted <sahmed1020-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

how to start the process

Gitted — 2012-04-17T13:42:23

How does godrb actually start, is there a ubuntu init script that
starts the process and makes sure it starts upon reboots?

Re: monitor chrome with god

Nuno Valente — 2012-04-06T19:25:41

Try adding :

      w.stop = "kill -9 {{PID}}"

      w.behavior(:clean_pid_file)

      w.start_if do |start|
        start.condition(:process_running) do |c|
          c.interval = 30.seconds
          c.running = false
        end
      end

Best regards,
Nuno Valente.

On Mon, Apr 2, 2012 at 2:20 PM, Sheng LUO <sheng.peisi.luo-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

[fail] never received process exit event

Toan Nguyen — 2012-04-06T03:36:28

Hi everybody,

I'm a newbie to God. I install and run God on Ubuntu system is fine.

But I get problem when run God on Debian. Run "god check", I got
message: "[fail] never received process exit event".

My system info:

No LSB modules are available.
Distributor ID:Debian
Description:Debian GNU/Linux 6.0.4 (squeeze)
Release:6.0.4
Codename:squeeze

Linux breithorn 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012
x86_64 GNU/Linux

I have checked cn (connector) kernel module by "lsmod | grep cn":

cn                      4563  0

I don't know why it can't check process exit event.

monitor chrome with god

Sheng LUO — 2012-04-02T13:20:40

hi there,

My ubuntu runs on 1G ram, where chrome always get fronzen or crashed 
because of flash.
So I want to watch my chrome with god and limit the cpu and memory it 
occupies.
Normally, I can run a chrome by typing "google-chrome" in terminal, so I 
write following god script:

God.watch do |w|
  w.name = "chrome"
  w.start = "/opt/google/chrome/chrome"
  w.keepalive :cpu_max => 80.percent, :memory_max => 500.megabytes
end

When I start the script above by "god -c chrome.god -D", it just keep 
opening new chrome.
Doesn't it automatically figure out there is already a chrome running?
What can I do to realize monitoring chrome?

Re: non-daemon process using Bundler

Christopher J. Bottaro — 2012-03-18T01:55:43

Hey Pete,

Thanks for the reply.  Surprisingly enough, I resolved this issue by 
letting god handle the log file.  In other words, I changed my god config 
like so:

  w.start     = "bundle exec rake resque:scheduler 
to
  w.start     = "bundle exec rake resque:scheduler"
  w.log       = "#{SHARED_PATH}/log/scheduler.log"

Not sure why that fixed it.  Probably has something to do with Bash/Unix 
process intricacies that I don't fully understand.

Re: non-daemon process using Bundler

Pete Shima — 2012-03-17T00:42:55

Hi Chris, 

I'd guess the issue is that god is starting your sh shell and the shell is starting ruby, rather than god.  I am not sure how god deals with processes like these but I'd guess the issue not to be with bundle exec.

I'd suggest running resque scheduler directly without wrapping in sh -c if your environment allows.

here's my w.start command that is confirmed to be working correctly under bundle exec

w.start = "#{BIN_PATH}/bundle exec rake -f #{RAILS_ROOT}/Rakefile resque:scheduler"

Hope this helps.


On Friday, March 16, 2012 at 11:36 AM, Christopher J. Bottaro wrote:

non-daemon process using Bundler

Christopher J. Bottaro — 2012-03-16T18:36:49

Hello,

I'm having a problem with god trying to watch a non-daemon process that is 
started by "bundle exec".

God starts the process just fine, as seen by ps:

6856 ?        Ss     0:00 sh -c bundle exec rake resque:scheduler >> 
/home/rails/project_a/shared/log/scheduler.log 2>&1
6860 ?        Sl     0:23 ruby 
/home/rails/project_a/shared/bundle/ruby/1.9.1/bin/rake resque:scheduler

The problem is that god is monitoring the first process (6856), so that a 
"god stop" or "god restart" nukes the first process, but the second one 
remains.

Any ideas?  Thanks for the help.

Re: God load without start

Christopher J. Bottaro — 2012-03-12T15:29:37

For the situation where you have a Chef provisioned server (including 
installing/configuring God), but then your app is deployed via Capistrano 
at a later time.

On Monday, December 13, 2010 9:14:27 AM UTC-5, Andrés Mejía wrote:

Test issues on Ruby 1.9.x and OS X Lion

Max Lansing — 2012-02-29T07:53:20

I seem to be unable to complete a rake run with 0.12.1, using Ruby 1.9.2 or 1.9.3 on OS 10.7.3.

It hangs at TestEventHandlerOperational: test_operational, in the God::EventHandler.start call. Within that method, it's hanging on ::Process.kill('KILL', pid). When kill is called, the child process is terminated, but sticks around as a zombie, and the kill call never returns.

The tests run normally for me in 1.8.7. 

'god check', which runs similar code, runs fine in 1.9.x.

Can anyone reproduce this issue?

Thanks,

Max

Killing a process doesn't restart

Matias — 2012-02-20T18:18:50

I create the follow solr.god: https://gist.github.com/1870445

The process start fine but if I kill the process, God doesn't restart
the process again. What I miss?

notification on process restart / crash when events are disabled

2012-02-22T16:06:32

Hi. I've got a couple of questions:

1) Is this project dead? Does it have an active maintainer who'll be
able to fix bugs?
2) How do I notify our ops team of process restarts or crashes when
god's running with user privileges, thus not having access to events
interface?

Thank you.

Re: Custom Notifications

Dan Cook — 2012-02-20T02:26:01

A little more context on this:
I set up a very simple test where god was using the above condition.

w.transition(:up, :start) do |on|
 on.condition(:process_exits) do |c|
   LOG.info("Hello world.")
 end
end

and the resulting output is:

I [2012-02-19 18:22:08]  INFO: Loading foo.god
I [2012-02-19 18:22:08]  INFO: Syslog enabled.
I [2012-02-19 18:22:08]  INFO: Using pid file directory: /var/run/god
I [2012-02-19 18:22:08]  INFO: Hello world.
I [2012-02-19 18:22:08]  INFO: Started on drbunix:///tmp/god.17165.sock

So the above code only runs once at start time so my original question
stands.

How do I register a custom notification hook in god?

Cheers,
Dan

Custom Notifications

Dan Cook — 2012-02-18T19:41:58

I need to execute a some ruby code on notification.
We have a central health monitoring service on the box and process
health is just one part of the overall reports.
I would like to put the events into a queue on the disk and have our
health monitoring service slurp up any events and send them to another
centralized server.

I assume I can just do this:

w.transition(:up, :start) do |on|
  on.condition(:process_exits) do |c|
    # custom notification method here....
  end
end

Is this correct?

Cheers,
Dan

rbenv shims

ileitch — 2012-02-16T15:16:34

Hi,

What's the correct approach for my God scripts to see my rbenv shims?

God itself starts fine, as I set PATH in /etc/init.d/god:

PATH=/root/.rbenv/shims:/root/.rbenv/bin:$PATH

The individual watchers however do not have the shims in the PATH. I
could just hard code the absolute path to the shim in my watcher, but
I feel there must be a cleaner way?

I also don't see an option for setting PATH in God.

Cheers
Ian