Monitor and Control Applications Using Supervisor: Part 1

Supervisor is a monitoring tool which controls various child processes and handles starting/restarting of these child processes when they exit abruptly or exit due to some reasons. It can be extended to control the processes via the XML-RPC API over remote locations without logging in to the server. I will cover the XML-RPC API in the second part of this two-part series.

In the first part of this tutorial series, I will take you through how to set up Supervisor to work with our application. For this, I will create a trivial application in Flask along with Gunicorn to act as our WSGI HTTP server. 

I assume that you have a basic understanding of Flask, Gunicorn and environment setup best practices using virtualenv to be followed while developing a Python application.

Installing Packages

The following packages need to be installed for developing and controlling the application we'll be developing.

$ pip install gunicorn supervisor Flask

The Trivial Application

I will now create a trivial application in Flask. This application is small but will meet the needs of this tutorial.

Below is the structure of the application:

flask_app/
    my_app/
        - __init__.py
        - views.py

flask_app/my_app/__init__.py

from flask import Flask
app = Flask(__name__)

import my_app.views

In the file above, the application has been configured and initialised.

flask_app/my_app/views.py

from my_app import app

@app.route('/')
def hello_world():
    return 'Hello to the World of Flask!'

In the file above, I have created a simple Hello World endpoint.

To run the application as a process, we can use Gunicorn. To check if Gunicorn is working as expected, just run the following command from inside the application folder.

$ gunicorn -w 4 -b 127.0.0.1:8000 my_app:app

After this, point your browser to http://127.0.0.1:8000/ to see the application's home page.

Configuring and Using Supervisor

Now, we need to do the same using Supervisor so that this runs as a daemon and will be controlled by Supervisor itself rather than human intervention. 

First of all, we need a Supervisor configuration file. Supervisor, by default, looks for an etc folder that has a file named supervisord.conf. In system-wide installations, this folder is /etc/, and in virtualenv, it will look for an etc folder in virtualenv and then fall back to /etc/.

In the case of a system-wide installation, run:

$ echo_supervisord_conf > /etc/supervisord.conf

In the case of virtualenv, from inside the virtualenv root folder, run:

$ echo_supervisord_conf > etc/supervisord.conf

The echo_supervisord_conf program is provided by Supervisor; it prints a sample config file to the location specified. This will create a file named supervisord.conf at the location specified in the above command.

To configure Supervisor for our application, add the following block somewhere in this file:

[program:supervisor_demo]
command=<path/to/virtualenv>/bin/gunicorn -w 4 -b 127.0.0.1:8000 my_app:app
directory=<path/to/virtualenv>/supervisor-tutsplus-demo
user=someuser # Relevant user
autostart=true
autorestart=true
stdout_logfile=/tmp/app.log
stderr_logfile=/tmp/error.log

Here we configure the command that needs to be run and from which directory location it needs to do so under which user. Also specified is whether to restart the application in case of any failure. The log files' locations can also be specified. There are many other configuration options, which can be looked up in the official documentation of Supervisor.

Make a note that one should never run the applications as a root user. This is a huge security flaw as the application crashes, which can harm the OS itself.

To run the application using Supervisor, run the following commands:

$ supervisord
$ supervisorctl status
supervisor_demo        RUNNING   pid 22550, uptime 0:00:04

The first command invokes the supervisord server, and the next one gives a status of all the child processes.

Every time you make a change to your application and then wish to restart Gunicorn in order for it to reflect the changes, run the following command:

$ supervisorctl restart all

You can also give specific processes instead of restarting everything: 

$ supervisorctl restart supervisor_demo

Supervisor also provides a Web UI interface which can be enabled by enabling the inet_http_server configuration block in supervisord.conf. This Web UI can be looked up on http://localhost:9001/.

Conclusion

In this tutorial, we have seen how to configure Supervisor and use it to run an application as a daemon process and control the processes collectively or selectively. In the second part of this tutorial series, we will see how to control and interrogate Supervisor and its programs/processes using the XML-RPC API.

by Shalabh Aggarwal via Tuts+ Code