Ir al contenido principal

Ralsina.Me — El sitio web de Roberto Alsina

Deploying Django Into My Cheap VPS

I am pre­pa­ring to open my cheap si­te-an­d-­blo­g-hos­ting ser­vi­ce to the pu­blic at so­me poin­t, so I nee­ded to do so­me groun­dwo­rk in­to de­plo­y­men­t. Con­si­der that the host that wi­ll run it wi­ll ha­ve ve­ry li­mited re­sour­ce­s, so I nee­ded to find lean and cheap so­lu­tions when po­s­si­ble, but at the sa­me ti­me, I want to achie­ve rea­so­na­ble re­lia­bi­li­ty and ea­se of de­plo­y­men­t.

Sin­ce this is a tes­ting ser­ve­r, I want it to ha­ve git mas­ter de­plo­ye­d. I do­n't want au­to­ma­tic de­plo­y­men­t, but I want to de­ploy often, mea­ning se­ve­ral ti­mes dai­l­y.

I pre­fe­rred sim­ple tools ins­tead of com­plex tool­s, li­gh­twe­ight tools wi­th just enou­gh fea­tu­res ins­tead of hea­vie­r, mo­re fu­ll­y-­fea­tu­red tool­s. Your choi­ces on ea­ch step could and pro­ba­bly should be di­ffe­rent than mi­ne, de­pen­ding on your si­tua­tio­n, re­qui­re­men­ts and per­so­nal pre­fe­ren­ce­s.

So, he­re's my no­tes from how it's do­ne cu­rren­tl­y. This is not meant as a HO­W­TO, just a des­crip­tion of what see­ms to be wo­rking we­ll enou­gh so fa­r.

Preparing the System

Along this de­plo­y­ment we'­ll use a lot of stu­ff. So, lo­ts of stu­ff nee­ds to be ins­ta­lled and con­fi­gu­red be­fo­re we can start de­plo­y­men­t.

  1. Get rid of all se­r­­vi­­ces that wi­­ll not be nee­­de­­d. Idea­­lly no­­­thing should be run­­ning on the sys­­tem for whi­­ch you do­­n't ha­­ve a co­m­­pe­­lling rea­­so­­n.

  2. In­s­­ta­­ll and se­­tup DBMS. I got MyS­­QL via the Ubun­­tu pa­­cka­­ge.

  3. In­s­­ta­­ll other things. In this ca­se, that mean­­t:

    • Vi­­r­­­tua­­­lenv

    • Re­­­dis (U­s­ed to crea­­­te job queues)

    • Ngi­nx (For re­­­ve­­r­­­se pro­­­­­x­­­yi­n­­­g)

    • Ga­­­tling (For sta­­­tic fi­­­le se­­r­­­vi­n­­­g)

    • mo­­­­­nit (For se­­r­­­vi­­­ce mo­­­­­ni­­­to­­­­­ri­n­­­g/­­­res­­­ta­­r­­­ti­n­­­g)

    • Exi­­­m4 (For SM­­­TP se­­r­­­vi­­­ce)

Choices made here:

Virtualenv instead of Ubuntu/Debian packages

The chan­ce to en­su­re I am ge­tting the exact sa­me ver­sion of eve­r­y­thing as in my de­ve­lo­p­ment ma­chi­ne makes this a no­-­bra­i­ne­r. Al­so, it means I can de­ploy this as a no­n-­root use­r.

MySQL instead of PostgreSQL

It's not a big deal for the tes­ting ser­ve­r, rea­ll­y. It could be SQ­Li­te and you may not even no­ti­ce.

Monit instead of Supervisor or Circus

I could­n't qui­te get Cir­cus+­Chausse­tte to wo­rk co­rrec­tl­y, and Su­per­vi­sor uses mo­re re­sour­ces than mo­ni­t. Mo­nit has a ni­ce web "con­trol pa­ne­l".

Exim4 instead of Sendmail, Postfix, or something else

No strong pre­fe­ren­ce, ex­cept ea­se of con­fi­gu­ra­tio­n. If I can make Djan­go use a send­mail bi­na­ry ins­tead of a SM­TP so­cket to send emai­l, then this is out and nu­ll­mai­ler is in, to avoid a lon­g-­li­ved pro­ce­ss.

Nginx instead of Apache

I heard good things, and Apa­che bo­res me, so why not try so­me­thing di­ffe­ren­t?

Gatling instead of Nginx

I al­ready had a Ga­tling se­tup he­re, and it's neat way to deal wi­th vir­tual do­main­s, plus low re­sour­ce usage ma­de me want to keep it. Al­so, it being a se­pa­ra­te ser­vi­ce from Ngi­nx means if Ngi­nx die­s, the si­tes wi­ll stay ali­ve.

Getting the Code Up There

Sin­ce the pro­ject is hos­ted at Gi­tHu­b, it's ve­ry rea­so­na­ble to just use gi­t. Sin­ce it's pu­re py­tho­n, vir­tua­lenv and pip are good for de­pen­den­cy hand­lin­g.

So, I created a user at the VP­S, and started wri­ting a script that should get eve­r­y­thing in pla­ce, pre­sen­ted he­re wi­th mu­ch mo­re co­m­ment that it has in real li­fe:

#/bin/sh
if [ ! -d nikola-server ]
then
    # This means this is an initial deployment or else I have nuked
    # everything. So start by cloning the repo and creating the
    # virtualenv it will use
    git clone git://github.com/ralsina/nikola-server.git
    virtualenv nikola-server/venv
fi

# Go into the repo and rollback any changes to settings.py
cd nikola-server
git checkout alva/alva/settings.py
# Update from master
git pull
# Override settings.py with the deployment version, which
# has things like the proper DB settings and such.
# Yes I know I could override it using an env. variable
# bu this works too.
cp ../settings.py alva/alva/settings.py

# Activate the venv
. venv/bin/activate

# Enter the django project's folder and install all it requires
cd alva
pip install -r requirements.txt

# These are requirements for deployment which the development
# setup doesn't need (MySQL driver, gunicorn)
pip install mysql-python
pip install gunicorn

# Bring the deployment server's DB up to speed
./manage.py syncdb
# Perform any necessary DB migrations
./manage.py migrate  allauth.socialaccount
./manage.py migrate allauth.socialaccount.providers.twitter
./manage.py migrate blogs

# Put all the static files in the right place so the right
# webserver will pick it up
./manage.py collectstatic --noinput

Choices made here:

A simple shell script instead of Puppet or Chef.

This is not com­pli­ca­ted stu­ff, it does­n't need mu­ch, a she­ll script is good enou­gh. I can alwa­ys grow it in­to a re­ci­pe la­ter on.

I wi­ll su­re­ly do that when/if this mo­ves from tes­ting in­to pro­duc­tio­n, spe­cia­lly sin­ce I wi­ll want to ha­ve all the ser­vi­ces con­fi­gu­red from a ver­sio­ned re­po.

Gunicorn instead of something else

I heard good things. No par­ti­cu­lar­ly strong pre­fe­ren­ce.

Web Servers

This de­plo­y­ment in­vol­ves a num­ber of web­ser­ver­s:

Gunicorn, running in localhost:9000

No con­fi­gu­ra­tio­n, it uses the co­m­mand li­ne.

Gatling, running in 184.82.108.14:80 serving static files:

Configured in /etc/default/gatling:

DAEMON_OPTS="-e -v -D -S -F -U -u nobody -c /var/www"
DAEMON_OPTS="-c /srv/www -P 2M -d  -v -p 80 -F -S -i 184.82.108.14"
DAEMON="gatling"
Nginx, running as reverse proxy for Gunicorn in 184.82.108.15:80:

Configured in /e­tc/n­gi­n­x/­si­tes-e­na­ble­d/­de­fault:

server {
        #listen   80; ## listen for ipv4; this line is default and implied
        #listen   [::]:80 default ipv6only=on; ## listen for ipv6

        listen 184.82.108.15:80;
        index index.html index.htm;

    server_name donewithniko.la;
    # no security problem here, since / is alway passed to upstream
    root /path/to/test/hello;
    location / {
        proxy_pass_header Server;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Scheme $scheme;
        proxy_connect_timeout 10;
        proxy_read_timeout 10;
        proxy_pass http://localhost:9000/;
    }
}

Decisions Made Here

Treating the Django app and the generated static sites as different services.

By chan­ce, this VPS co­mes wi­th two IP addresses, whi­ch means I can use di­ffe­rent so­ftwa­re in port 80. I took ad­van­ta­ge of that to treat this as two com­ple­te­ly se­pa­ra­te pro­duc­ts.

The rea­son is that it's po­s­si­ble that it makes mo­re sen­se to ha­ve two ac­tual ser­ver­s, one for the Djan­go app and one for ser­ving the ge­ne­ra­ted sta­tic si­tes, and I wanted to make un­com­pro­mi­s­ed de­ci­sions for ea­ch, and to ha­ve the chan­ce to try them and tweak the­m.

Running Things

Things should start and stay started. Process should not run as root if not needed. Other than that, I have no strong requirements. Here's the monit configuration (/etc/monit/conf.d/alva.conf) , which should be fairly self-explanatory except for the startstop.sh thing explained in the sidebar:

check process gunicorn with pidfile /home/alva/gunicorn.pid
    start program = "/bin/startstop.sh start gunicorn alva /usr/bin/writelog /home/alva/gunicorn.log /home/deployer/nikola-server/venv/bin/python /home/deployer/nikola-server/alva/manage.py run_gunicorn --bind=127.0.0.1:9000"
    stop program = "/bin/startstop.sh stop gunicorn"
    if failed host 127.0.0.1 port 9000
        protocol http then restart

check process rqworker with pidfile /home/alva/rqworker.pid
    start program = "/bin/startstop.sh start rqworker alva /usr/bin/writelog /home/alva/rqworker.log /home/deployer/nikola-server/venv/bin/python /home/deployer/nikola-server/alva/manage.py rqworker"
    stop program = "/bin/startstop.sh stop rqworker"

check process nginx with pidfile /var/run/nginx.pid
    start program = "/etc/init.d/nginx start"
    stop program = "/etc/init.d/nginx stop"
    if failed host 184.82.108.15 port 80
        protocol http then restart

check process gatling with pidfile /var/run/gatling.pid
    start program = "/etc/init.d/gatling start"
    stop program = "/etc/init.d/gatling stop"
    if failed host 184.82.108.14 port 80
        protocol http then restart

check process redis with pidfile /var/run/redis.pid
    start program = "/etc/init.d/redis-server start"
    stop program = "/etc/init.d/redis-server stop"

check exim4 with pidfile /var/run/exim4/exim.pid
    start program = "/etc/init.d/exim4 start"
    stop program = "/etc/init.d/exim4 stop"
    if failed host 127.0.0.1 port 25 then restart

check mysql with pidfile /var/run/mysqld/mysqld.pid
    start program = "/etc/init.d/mysql start"
    stop program = "/etc/init.d/mysql stop"
    if failed host 127.0.0.1 port 3306 then restart

So, that star­ts eve­r­y­thing I want run­nin­g, and ho­pe­fu­lly wi­ll keep it run­nin­g.

How To Deploy

First a bit of CLI ac­tio­n:

ssh -l deployer burst.ralsina.me
sh -x deploy.sh

Then, via mo­ni­t, res­tart gu­ni­corn and rqwo­rke­r. And tha­t's it. The­re's sti­ll so­me things that could be do­ne to make it mo­re strea­m­li­ned but it's good enou­gh at this poin­t.

Jose / 2013-06-03 17:49:

Si tenés que usar MySQL con InnoDB como motor de almacenamiento te sugiero que uses la versión 5.6x, o mejor aún MariaDB o las versiones de Percona[1] de MySQL. Cualquiera de las 2 últimas le pegan un par de vueltas a la manzana al MySQL de Oracle.

Si vas a usar MyISAM como motor de almacenamiento, olvidate de todo lo que te dije :P

[1] http://www.percona.com/soft...

Roberto Alsina / 2013-06-03 17:58:

Justo este proveedor de VPS me da un ubuntu jovato que tiene MySQL 5.1.69 :(

Jose / 2013-06-03 18:05:

Uff.. viejecito... si algún día tenés que jubilarlo porque ya le agarró azheimer acá tenés como hacerlo:

http://www.percona.com/doc/...
https://downloads.mariadb.o...

Nassty / 2013-06-03 20:27:

Hola Roberto.

No probe con gunicorn, pero hace un tiempo hice un benchmark de uwsgi corriendo sobre un socket TCP y un socket Unix, y consegui aumentar la velocidad de respuesta en un ~3%. No es mucho pero todo suma!

Roberto Alsina / 2013-06-03 20:28:

Tambien tendria que ponerme a ver el benchmark que hay en la pagina de gunicorn para elegir entre gevent y todos los otros backends que soporta.

Martín Gaitán / 2013-06-03 20:30:

Pegale una mirada a esto: https://github.com/jonathan... . Para mi le pasa el trapo a Fabric (que para mi es bash escrito en python. it sucks)

Roberto Alsina / 2013-06-03 20:36:

Se ve muy piola! Me parece que sale secuela...


Contents © 2000-2020 Roberto Alsina