Marat's Notes

Using Monit with Rails stack

Dec 21, 2018
2 minute read

Monit documentation - https://mmonit.com/monit/documentation/monit.html

How Monit works?

Monit - super lightweight tool to monitor your processes and services.

You can also use Monit to monitor files, directories and filesystems. Monit can monitor these items for changes, such as timestamps changes, checksum changes or size changes.

This is also useful for security reasons - you can monitor the md5 or sha1 checksum of files that should not change and get an alert or perform an action if they should change.

Also Monit can be used to monitor network connections and general system resources such as overall cpu usage, memory and system load.

Systemd script for sidekiq

Let’s imagine, that we want to add monitoring for our Sidekiq instance on one of our servers.

[Unit]
Description=Sidekiq-sync
After=multi-user.target

[Service]
Environment=RAILS_ENV=production
EnvironmentFile=/etc/environment
WorkingDirectory=/home/deploy/apps/myapp/current/
ExecStart=/home/deploy/apps/myapp/current/bin/bundle exec sidekiq -e production -L /home/deploy/sidekiq-sync.log -C /home/deploy/apps/myapp/current/config/sidekiq-sync.yml -P /home/deploy/apps/myapp/shared/tmp/pids/sidekiq_sync.pid
User=deploy
Group=deploy
UMask=0002
RestartSec=1
Restart=on-failure

[Install]
WantedBy=multi-user.target

Monit configuration

Let’s check configuration file /etc/monit/monitrc.

We want to use webUI and add login and password to use basic http auth.

set httpd port 2812 and
    #use address localhost  # only accept connection from localhost
    #allow 12.0.0.1        # allow localhost to connect to the server and
    allow root:password      # require user 'test' with password 'test'

Cool, let’s check config file first, it’s easy like Nginx config test:

root@4vcpu-8gb-fra1-01:/home/deploy# monit -t
Control file syntax OK

Let’s create /etc/monit/conf.d/sidekiq_sync:

check process sidekiq_sync
  with pidfile "/home/deploy/apps/myapp/shared/tmp/pids/sidekiq_sync.pid"
  start program = "/bin/systemctl start sidekiq_sync" with timeout 60 seconds
  stop program = "/bin/systemctl stop sidekiq_sync"
  if does not exist for 1 cycle
    then exec "/usr/local/bin/monit2slack --webhook 'https://hooks.slack.com/services/XXX/XXXX/XXX' --service Sidekiq_Sync --status error --text 'Process Sidekiq Sync is down'"
    else if succeeded for 1 cycle then exec "/usr/local/bin/monit2slack --webhook 'https://hooks.slack.com/services/XXX/BEEGG3M1N/XXX' --service Sidekiq_Sync --status error --text 'Process Sidekiq Sync is up'"
  if does not exist then restart

Here we added Slack message on Stop and Start events, if for example we stop Sidekiq process with killing it by PID, Monit detect it and try to start process again.

Use webUI, Luke!

Let’s point our browser to the our server IP at port 2812:

monit_web

Nice! Everything works. Also we can check basic system metrics:

monit_web2

And for example selected process info, also we can start/stop/restart service from webUI or disable monitoring.

monit_web3