Skip to main content

Ralsina.Me — Roberto Alsina's website

Deploying Django Into My Cheap VPS

I am pre­par­ing to open my cheap site-and-blog-host­ing ser­vice to the pub­lic at some point, so I need­ed to do some ground­work in­to de­ploy­men­t. Con­sid­er that the host that will run it will have very lim­it­ed re­sources, so I need­ed to find lean and cheap so­lu­tions when pos­si­ble, but at the same time, I want to achieve rea­son­able re­li­a­bil­i­ty and ease of de­ploy­men­t.

Since this is a test­ing server, I want it to have git mas­ter de­ployed. I don't want au­to­mat­ic de­ploy­men­t, but I want to de­ploy of­ten, mean­ing sev­er­al times dai­ly.

I pre­ferred sim­ple tools in­stead of com­plex tool­s, light­weight tools with just enough fea­tures in­stead of heav­ier, more ful­ly-fea­tured tool­s. Your choic­es on each step could and prob­a­bly should be dif­fer­ent than mine, de­pend­ing on your sit­u­a­tion, re­quire­ments and per­son­al pref­er­ences.

So, here's my notes from how it's done cur­rent­ly. This is not meant as a HOW­TO, just a de­scrip­tion of what seems to be work­ing well enough so far.

Preparing the System

Along this de­ploy­ment we'll use a lot of stuff. So, lots of stuff needs to be in­stalled and con­fig­ured be­fore we can start de­ploy­men­t.

  1. Get rid of all ser­vices that will not be need­ed. Ide­al­­ly noth­ing should be run­n­ing on the sys­tem for which you don't have a com­pelling rea­­son.

  2. In­­stall and set­up DB­M­S. I got MySQL via the Ubun­­tu pack­­age.

  3. In­­stall oth­­er things. In this case, that mean­t:

    • Vir­­­tualenv

    • Re­dis (Used to cre­ate job queues)

    • Ng­inx (For re­­­verse prox­­­y­ing)

    • Gatling (For stat­ic file serv­ing)

    • monit (For ser­vice mon­i­­­tor­ing/restart­ing)

    • Ex­im4 (For SMTP ser­vice)

Choices made here:

Virtualenv instead of Ubuntu/Debian packages

The chance to en­sure I am get­ting the ex­act same ver­sion of ev­ery­thing as in my de­vel­op­ment ma­chine makes this a no-brain­er. Al­so, it means I can de­ploy this as a non-­root us­er.

MySQL instead of PostgreSQL

It's not a big deal for the test­ing server, re­al­ly. It could be SQLite and you may not even no­tice.

Monit instead of Supervisor or Circus

I could­n't quite get Cir­cus+Chaus­sette to work cor­rect­ly, and Su­per­vi­sor us­es more re­sources than monit. Monit has a nice web "con­trol pan­el".

Exim4 instead of Sendmail, Postfix, or something else

No strong pref­er­ence, ex­cept ease of con­fig­u­ra­tion. If I can make Djan­go use a send­mail bi­na­ry in­stead of a SMTP sock­et to send email, then this is out and null­mail­er is in, to avoid a long-lived process.

Nginx instead of Apache

I heard good things, and Apache bores me, so why not try some­thing dif­fer­en­t?

Gatling instead of Nginx

I al­ready had a Gatling set­up here, and it's neat way to deal with vir­tu­al do­main­s, plus low re­source us­age made me want to keep it. Al­so, it be­ing a sep­a­rate ser­vice from Ng­inx means if Ng­inx dies, the sites will stay alive.

Getting the Code Up There

Since the project is host­ed at GitHub, it's very rea­son­able to just use git. Since it's pure python, vir­tualenv and pip are good for de­pen­den­cy han­dling.

So, I cre­at­ed a us­er at the VP­S, and start­ed writ­ing a script that should get ev­ery­thing in place, pre­sent­ed here with much more com­ment that it has in re­al life:

#/bin/sh
if [ ! -d nikola-server ]
then
    # This means this is an initial deployment or else I have nuked
    # everything. So start by cloning the repo and creating the
    # virtualenv it will use
    git clone git://github.com/ralsina/nikola-server.git
    virtualenv nikola-server/venv
fi

# Go into the repo and rollback any changes to settings.py
cd nikola-server
git checkout alva/alva/settings.py
# Update from master
git pull
# Override settings.py with the deployment version, which
# has things like the proper DB settings and such.
# Yes I know I could override it using an env. variable
# bu this works too.
cp ../settings.py alva/alva/settings.py

# Activate the venv
. venv/bin/activate

# Enter the django project's folder and install all it requires
cd alva
pip install -r requirements.txt

# These are requirements for deployment which the development
# setup doesn't need (MySQL driver, gunicorn)
pip install mysql-python
pip install gunicorn

# Bring the deployment server's DB up to speed
./manage.py syncdb
# Perform any necessary DB migrations
./manage.py migrate  allauth.socialaccount
./manage.py migrate allauth.socialaccount.providers.twitter
./manage.py migrate blogs

# Put all the static files in the right place so the right
# webserver will pick it up
./manage.py collectstatic --noinput

Choices made here:

A simple shell script instead of Puppet or Chef.

This is not com­pli­cat­ed stuff, it does­n't need much, a shell script is good enough. I can al­ways grow it in­to a recipe lat­er on.

I will sure­ly do that when/if this moves from test­ing in­to pro­duc­tion, spe­cial­ly since I will want to have all the ser­vices con­fig­ured from a ver­sioned re­po.

Gunicorn instead of something else

I heard good things. No par­tic­u­lar­ly strong pref­er­ence.

Web Servers

This de­ploy­ment in­volves a num­ber of web­server­s:

Gunicorn, running in localhost:9000

No con­fig­u­ra­tion, it us­es the com­mand line.

Gatling, running in 184.82.108.14:80 serving static files:

Configured in /etc/default/gatling:

DAEMON_OPTS="-e -v -D -S -F -U -u nobody -c /var/www"
DAEMON_OPTS="-c /srv/www -P 2M -d  -v -p 80 -F -S -i 184.82.108.14"
DAEMON="gatling"
Nginx, running as reverse proxy for Gunicorn in 184.82.108.15:80:

Configured in /etc/ng­inx/sites-en­abled/de­fault:

server {
        #listen   80; ## listen for ipv4; this line is default and implied
        #listen   [::]:80 default ipv6only=on; ## listen for ipv6

        listen 184.82.108.15:80;
        index index.html index.htm;

    server_name donewithniko.la;
    # no security problem here, since / is alway passed to upstream
    root /path/to/test/hello;
    location / {
        proxy_pass_header Server;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Scheme $scheme;
        proxy_connect_timeout 10;
        proxy_read_timeout 10;
        proxy_pass http://localhost:9000/;
    }
}

Decisions Made Here

Treating the Django app and the generated static sites as different services.

By chance, this VPS comes with two IP ad­dress­es, which means I can use dif­fer­ent soft­ware in port 80. I took ad­van­tage of that to treat this as two com­plete­ly sep­a­rate prod­uct­s.

The rea­son is that it's pos­si­ble that it makes more sense to have two ac­tu­al server­s, one for the Djan­go app and one for serv­ing the gen­er­at­ed stat­ic sites, and I want­ed to make un­com­pro­mised de­ci­sions for each, and to have the chance to try them and tweak them.

Running Things

Things should start and stay started. Process should not run as root if not needed. Other than that, I have no strong requirements. Here's the monit configuration (/etc/monit/conf.d/alva.conf) , which should be fairly self-explanatory except for the startstop.sh thing explained in the sidebar:

check process gunicorn with pidfile /home/alva/gunicorn.pid
    start program = "/bin/startstop.sh start gunicorn alva /usr/bin/writelog /home/alva/gunicorn.log /home/deployer/nikola-server/venv/bin/python /home/deployer/nikola-server/alva/manage.py run_gunicorn --bind=127.0.0.1:9000"
    stop program = "/bin/startstop.sh stop gunicorn"
    if failed host 127.0.0.1 port 9000
        protocol http then restart

check process rqworker with pidfile /home/alva/rqworker.pid
    start program = "/bin/startstop.sh start rqworker alva /usr/bin/writelog /home/alva/rqworker.log /home/deployer/nikola-server/venv/bin/python /home/deployer/nikola-server/alva/manage.py rqworker"
    stop program = "/bin/startstop.sh stop rqworker"

check process nginx with pidfile /var/run/nginx.pid
    start program = "/etc/init.d/nginx start"
    stop program = "/etc/init.d/nginx stop"
    if failed host 184.82.108.15 port 80
        protocol http then restart

check process gatling with pidfile /var/run/gatling.pid
    start program = "/etc/init.d/gatling start"
    stop program = "/etc/init.d/gatling stop"
    if failed host 184.82.108.14 port 80
        protocol http then restart

check process redis with pidfile /var/run/redis.pid
    start program = "/etc/init.d/redis-server start"
    stop program = "/etc/init.d/redis-server stop"

check exim4 with pidfile /var/run/exim4/exim.pid
    start program = "/etc/init.d/exim4 start"
    stop program = "/etc/init.d/exim4 stop"
    if failed host 127.0.0.1 port 25 then restart

check mysql with pidfile /var/run/mysqld/mysqld.pid
    start program = "/etc/init.d/mysql start"
    stop program = "/etc/init.d/mysql stop"
    if failed host 127.0.0.1 port 3306 then restart

So, that starts ev­ery­thing I want run­ning, and hope­ful­ly will keep it run­ning.

How To Deploy

First a bit of CLI ac­tion:

ssh -l deployer burst.ralsina.me
sh -x deploy.sh

Then, via monit, restart gu­ni­corn and rq­work­er. And that's it. There's still some things that could be done to make it more stream­lined but it's good enough at this point.

Jose / 2013-06-03 17:49:

Si tenés que usar MySQL con InnoDB como motor de almacenamiento te sugiero que uses la versión 5.6x, o mejor aún MariaDB o las versiones de Percona[1] de MySQL. Cualquiera de las 2 últimas le pegan un par de vueltas a la manzana al MySQL de Oracle.

Si vas a usar MyISAM como motor de almacenamiento, olvidate de todo lo que te dije :P

[1] http://www.percona.com/soft...

Roberto Alsina / 2013-06-03 17:58:

Justo este proveedor de VPS me da un ubuntu jovato que tiene MySQL 5.1.69 :(

Jose / 2013-06-03 18:05:

Uff.. viejecito... si algún día tenés que jubilarlo porque ya le agarró azheimer acá tenés como hacerlo:

http://www.percona.com/doc/...
https://downloads.mariadb.o...

Nassty / 2013-06-03 20:27:

Hola Roberto.

No probe con gunicorn, pero hace un tiempo hice un benchmark de uwsgi corriendo sobre un socket TCP y un socket Unix, y consegui aumentar la velocidad de respuesta en un ~3%. No es mucho pero todo suma!

Roberto Alsina / 2013-06-03 20:28:

Tambien tendria que ponerme a ver el benchmark que hay en la pagina de gunicorn para elegir entre gevent y todos los otros backends que soporta.

Martín Gaitán / 2013-06-03 20:30:

Pegale una mirada a esto: https://github.com/jonathan... . Para mi le pasa el trapo a Fabric (que para mi es bash escrito en python. it sucks)

Roberto Alsina / 2013-06-03 20:36:

Se ve muy piola! Me parece que sale secuela...


Contents © 2000-2024 Roberto Alsina