Preparing the System

Along this de­ploy­ment we'll use a lot of stuff. So, lots of stuff needs to be in­stalled and con­fig­ured be­fore we can start de­ploy­men­t.

1. Get rid of all ser­vices that will not be need­ed. Ide­al­­ly noth­ing should be run­n­ing on the sys­tem for which you don't have a com­pelling rea­­son.

2. In­­stall and set­up DB­M­S. I got MySQL via the Ubun­­tu pack­­age.

3. In­­stall oth­­er things. In this case, that mean­t:

   • Vir­­­tualenv

   • Re­dis (Used to cre­ate job queues)

   • Ng­inx (For re­­­verse prox­­­y­ing)

   • Gatling (For stat­ic file serv­ing)

   • monit (For ser­vice mon­i­­­tor­ing/restart­ing)

   • Ex­im4 (For SMTP ser­vice)

Getting the Code Up There

Since the project is host­ed at GitHub, it's very rea­son­able to just use git. Since it's pure python, vir­tualenv and pip are good for de­pen­den­cy han­dling.

So, I cre­at­ed a us­er at the VP­S, and start­ed writ­ing a script that should get ev­ery­thing in place, pre­sent­ed here with much more com­ment that it has in re­al life:

#/bin/sh
if [ ! -d nikola-server ]
then
    # This means this is an initial deployment or else I have nuked
    # everything. So start by cloning the repo and creating the
    # virtualenv it will use
    git clone git://github.com/ralsina/nikola-server.git
    virtualenv nikola-server/venv
fi

# Go into the repo and rollback any changes to settings.py
cd nikola-server
git checkout alva/alva/settings.py
# Update from master
git pull
# Override settings.py with the deployment version, which
# has things like the proper DB settings and such.
# Yes I know I could override it using an env. variable
# bu this works too.
cp ../settings.py alva/alva/settings.py

# Activate the venv
. venv/bin/activate

# Enter the django project's folder and install all it requires
cd alva
pip install -r requirements.txt

# These are requirements for deployment which the development
# setup doesn't need (MySQL driver, gunicorn)
pip install mysql-python
pip install gunicorn

# Bring the deployment server's DB up to speed
./manage.py syncdb
# Perform any necessary DB migrations
./manage.py migrate  allauth.socialaccount
./manage.py migrate allauth.socialaccount.providers.twitter
./manage.py migrate blogs

# Put all the static files in the right place so the right
# webserver will pick it up
./manage.py collectstatic --noinput

Web Servers

This de­ploy­ment in­volves a num­ber of web­server­s:

Gunicorn, running in localhost:9000

No con­fig­u­ra­tion, it us­es the com­mand line.

Gatling, running in 184.82.108.14:80 serving static files:

Configured in /etc/default/gatling:

DAEMON_OPTS="-e -v -D -S -F -U -u nobody -c /var/www"
DAEMON_OPTS="-c /srv/www -P 2M -d  -v -p 80 -F -S -i 184.82.108.14"
DAEMON="gatling"

Nginx, running as reverse proxy for Gunicorn in 184.82.108.15:80:

Configured in /etc/ng­inx/sites-en­abled/de­fault:

server {
        #listen   80; ## listen for ipv4; this line is default and implied
        #listen   [::]:80 default ipv6only=on; ## listen for ipv6

        listen 184.82.108.15:80;
        index index.html index.htm;

    server_name donewithniko.la;
    # no security problem here, since / is alway passed to upstream
    root /path/to/test/hello;
    location / {
        proxy_pass_header Server;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Scheme $scheme;
        proxy_connect_timeout 10;
        proxy_read_timeout 10;
        proxy_pass http://localhost:9000/;
    }
}

Running Things

Things should start and stay started. Process should not run as root if not needed. Other than that, I have no strong requirements. Here's the monit configuration (/etc/monit/conf.d/alva.conf) , which should be fairly self-explanatory except for the startstop.sh thing explained in the sidebar:

check process gunicorn with pidfile /home/alva/gunicorn.pid
    start program = "/bin/startstop.sh start gunicorn alva /usr/bin/writelog /home/alva/gunicorn.log /home/deployer/nikola-server/venv/bin/python /home/deployer/nikola-server/alva/manage.py run_gunicorn --bind=127.0.0.1:9000"
    stop program = "/bin/startstop.sh stop gunicorn"
    if failed host 127.0.0.1 port 9000
        protocol http then restart

check process rqworker with pidfile /home/alva/rqworker.pid
    start program = "/bin/startstop.sh start rqworker alva /usr/bin/writelog /home/alva/rqworker.log /home/deployer/nikola-server/venv/bin/python /home/deployer/nikola-server/alva/manage.py rqworker"
    stop program = "/bin/startstop.sh stop rqworker"

check process nginx with pidfile /var/run/nginx.pid
    start program = "/etc/init.d/nginx start"
    stop program = "/etc/init.d/nginx stop"
    if failed host 184.82.108.15 port 80
        protocol http then restart

check process gatling with pidfile /var/run/gatling.pid
    start program = "/etc/init.d/gatling start"
    stop program = "/etc/init.d/gatling stop"
    if failed host 184.82.108.14 port 80
        protocol http then restart

check process redis with pidfile /var/run/redis.pid
    start program = "/etc/init.d/redis-server start"
    stop program = "/etc/init.d/redis-server stop"

check exim4 with pidfile /var/run/exim4/exim.pid
    start program = "/etc/init.d/exim4 start"
    stop program = "/etc/init.d/exim4 stop"
    if failed host 127.0.0.1 port 25 then restart

check mysql with pidfile /var/run/mysqld/mysqld.pid
    start program = "/etc/init.d/mysql start"
    stop program = "/etc/init.d/mysql stop"
    if failed host 127.0.0.1 port 3306 then restart

So, that starts ev­ery­thing I want run­ning, and hope­ful­ly will keep it run­ning.

How To Deploy

First a bit of CLI ac­tion:

ssh -l deployer burst.ralsina.me
sh -x deploy.sh

Then, via monit, restart gu­ni­corn and rq­work­er. And that's it. There's still some things that could be done to make it more stream­lined but it's good enough at this point.