Deploying Django Into My Cheap VPS

I am preparing to open my cheap site-and-blog-hosting service to the public at some point, so I needed to do some groundwork into deployment. Consider that the host that will run it will have very limited resources, so I needed to find lean and cheap solutions when possible, but at the same time, I want to achieve reasonable reliability and ease of deployment.

Since this is a testing server, I want it to have git master deployed. I don't want automatic deployment, but I want to deploy often, meaning several times daily.

I preferred simple tools instead of complex tools, lightweight tools with just enough features instead of heavier, more fully-featured tools. Your choices on each step could and probably should be different than mine, depending on your situation, requirements and personal preferences.

So, here's my notes from how it's done currently. This is not meant as a HOWTO, just a description of what seems to be working well enough so far.

Preparing the System

Along this deployment we'll use a lot of stuff. So, lots of stuff needs to be installed and configured before we can start deployment.

  1. Get rid of all services that will not be needed. Ideally nothing should be running on the system for which you don't have a compelling reason.
  2. Install and setup DBMS. I got MySQL via the Ubuntu package.
  3. Install other things. In this case, that meant:
    • Virtualenv
    • Redis (Used to create job queues)
    • Nginx (For reverse proxying)
    • Gatling (For static file serving)
    • monit (For service monitoring/restarting)
    • Exim4 (For SMTP service)

Choices made here:

Virtualenv instead of Ubuntu/Debian packages
The chance to ensure I am getting the exact same version of everything as in my development machine makes this a no-brainer. Also, it means I can deploy this as a non-root user.
MySQL instead of PostgreSQL
It's not a big deal for the testing server, really. It could be SQLite and you may not even notice.
Monit instead of Supervisor or Circus
I couldn't quite get Circus+Chaussette to work correctly, and Supervisor uses more resources than monit. Monit has a nice web "control panel".
Exim4 instead of Sendmail, Postfix, or something else
No strong preference, except ease of configuration. If I can make Django use a sendmail binary instead of a SMTP socket to send email, then this is out and nullmailer is in, to avoid a long-lived process.
Nginx instead of Apache
I heard good things, and Apache bores me, so why not try something different?
Gatling instead of Nginx
I already had a Gatling setup here, and it's neat way to deal with virtual domains, plus low resource usage made me want to keep it. Also, it being a separate service from Nginx means if Nginx dies, the sites will stay alive.

Getting the Code Up There

Since the project is hosted at GitHub, it's very reasonable to just use git. Since it's pure python, virtualenv and pip are good for dependency handling.

So, I created a user at the VPS, and started writing a script that should get everything in place, presented here with much more comment that it has in real life:

#/bin/sh
if [ ! -d nikola-server ]
then
    # This means this is an initial deployment or else I have nuked
    # everything. So start by cloning the repo and creating the
    # virtualenv it will use
    git clone git://github.com/ralsina/nikola-server.git
    virtualenv nikola-server/venv
fi

# Go into the repo and rollback any changes to settings.py
cd nikola-server
git checkout alva/alva/settings.py
# Update from master
git pull
# Override settings.py with the deployment version, which
# has things like the proper DB settings and such.
# Yes I know I could override it using an env. variable
# bu this works too.
cp ../settings.py alva/alva/settings.py

# Activate the venv
. venv/bin/activate

# Enter the django project's folder and install all it requires
cd alva
pip install -r requirements.txt

# These are requirements for deployment which the development
# setup doesn't need (MySQL driver, gunicorn)
pip install mysql-python
pip install gunicorn

# Bring the deployment server's DB up to speed
./manage.py syncdb
# Perform any necessary DB migrations
./manage.py migrate  allauth.socialaccount
./manage.py migrate allauth.socialaccount.providers.twitter
./manage.py migrate blogs

# Put all the static files in the right place so the right
# webserver will pick it up
./manage.py collectstatic --noinput

Choices made here:

A simple shell script instead of Puppet or Chef.

This is not complicated stuff, it doesn't need much, a shell script is good enough. I can always grow it into a recipe later on.

I will surely do that when/if this moves from testing into production, specially since I will want to have all the services configured from a versioned repo.

Gunicorn instead of something else
I heard good things. No particularly strong preference.

Web Servers

This deployment involves a number of webservers:

Gunicorn, running in localhost:9000
No configuration, it uses the command line.
Gatling, running in 184.82.108.14:80 serving static files:

Configured in /etc/default/gatling:

DAEMON_OPTS="-e -v -D -S -F -U -u nobody -c /var/www"
DAEMON_OPTS="-c /srv/www -P 2M -d  -v -p 80 -F -S -i 184.82.108.14"
DAEMON="gatling"
Nginx, running as reverse proxy for Gunicorn in 184.82.108.15:80:

Configured in /etc/nginx/sites-enabled/default:

server {
        #listen   80; ## listen for ipv4; this line is default and implied
        #listen   [::]:80 default ipv6only=on; ## listen for ipv6

        listen 184.82.108.15:80;
        index index.html index.htm;

    server_name donewithniko.la;
    # no security problem here, since / is alway passed to upstream
    root /path/to/test/hello;
    location / {
        proxy_pass_header Server;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Scheme $scheme;
        proxy_connect_timeout 10;
        proxy_read_timeout 10;
        proxy_pass http://localhost:9000/;
    }
}

Decisions Made Here

Treating the Django app and the generated static sites as different services.

By chance, this VPS comes with two IP addresses, which means I can use different software in port 80. I took advantage of that to treat this as two completely separate products.

The reason is that it's possible that it makes more sense to have two actual servers, one for the Django app and one for serving the generated static sites, and I wanted to make uncompromised decisions for each, and to have the chance to try them and tweak them.

Running Things

Things should start and stay started. Process should not run as root if not needed. Other than that, I have no strong requirements. Here's the monit configuration (/etc/monit/conf.d/alva.conf) , which should be fairly self-explanatory except for the startstop.sh thing explained in the sidebar:

check process gunicorn with pidfile /home/alva/gunicorn.pid
    start program = "/bin/startstop.sh start gunicorn alva /usr/bin/writelog /home/alva/gunicorn.log /home/deployer/nikola-server/venv/bin/python /home/deployer/nikola-server/alva/manage.py run_gunicorn --bind=127.0.0.1:9000"
    stop program = "/bin/startstop.sh stop gunicorn"
    if failed host 127.0.0.1 port 9000
        protocol http then restart

check process rqworker with pidfile /home/alva/rqworker.pid
    start program = "/bin/startstop.sh start rqworker alva /usr/bin/writelog /home/alva/rqworker.log /home/deployer/nikola-server/venv/bin/python /home/deployer/nikola-server/alva/manage.py rqworker"
    stop program = "/bin/startstop.sh stop rqworker"

check process nginx with pidfile /var/run/nginx.pid
    start program = "/etc/init.d/nginx start"
    stop program = "/etc/init.d/nginx stop"
    if failed host 184.82.108.15 port 80
        protocol http then restart

check process gatling with pidfile /var/run/gatling.pid
    start program = "/etc/init.d/gatling start"
    stop program = "/etc/init.d/gatling stop"
    if failed host 184.82.108.14 port 80
        protocol http then restart

check process redis with pidfile /var/run/redis.pid
    start program = "/etc/init.d/redis-server start"
    stop program = "/etc/init.d/redis-server stop"

check exim4 with pidfile /var/run/exim4/exim.pid
    start program = "/etc/init.d/exim4 start"
    stop program = "/etc/init.d/exim4 stop"
    if failed host 127.0.0.1 port 25 then restart

check mysql with pidfile /var/run/mysqld/mysqld.pid
    start program = "/etc/init.d/mysql start"
    stop program = "/etc/init.d/mysql stop"
    if failed host 127.0.0.1 port 3306 then restart

So, that starts everything I want running, and hopefully will keep it running.

How To Deploy

First a bit of CLI action:

ssh -l deployer burst.ralsina.me
sh -x deploy.sh

Then, via monit, restart gunicorn and rqworker. And that's it. There's still some things that could be done to make it more streamlined but it's good enough at this point.

Comments

Comments powered by Disqus