I am preparing to open my cheap site-and-blog-hosting service to the public at some point, so I needed to do some groundwork into deployment. Consider that the host that will run it will have very limited resources, so I needed to find lean and cheap solutions when possible, but at the same time, I want to achieve reasonable reliability and ease of deployment.
Since this is a testing server, I want it to have git master deployed. I don't want automatic deployment, but I want to deploy often, meaning several times daily.
I preferred simple tools instead of complex tools, lightweight tools with just enough features instead of heavier, more fully-featured tools. Your choices on each step could and probably should be different than mine, depending on your situation, requirements and personal preferences.
So, here's my notes from how it's done currently. This is not meant as a HOWTO, just a description of what seems to be working well enough so far.
Working late last night in Alva I wanted to do something that sounded trivial:
When the page loads, get the current date and time, and if a certain input is empty, put it there like this:
So, how hard can that be, right? Well not hard, but...
Getting the current date-time is easy:
now = new Date();
have a untested, perhaps broken, limited version of it. And I am not about to add a strftime implementation to use it once.
Sure, there are a number of Date methods that convert to strings, but none of them lets you specify the output format.
To get the elements I want to put in the value, I used accessor methods. So, obviously, these should give me what I want for the string, right?
now.getDay(), now.getMonth(), now.getYear(), now.getHour() now.getMinute()
Well, they are, at the date mentioned above, respectively: 2, 4, 113, error, error
Ok, the errors are easy to fix from the docs. It's actually
getMinutes(), so now we have 2, 4, 113, 23, 45 and of those five things, the last two are what one would expect, at least. Let's go over the other three and see why they are so weird:
Date.getDay()returned 2 instead of 28
getDay()gives you the week day and not the day of the month. Which is absolutely idiotic. So, you have to use
getDate()instead. Which means the name is a lie, becasue the logical thing for
getDate()to return is the whole date.
Date.getMonth()returned 4 instead of 5
Date.getYear()returned 113 instead of 2013
Because it uses offset-from-1900. Which is amazing, and I had never heard of a language doing in a standard type. Because why? So, use
So, this produces that I want:
Unless... the day or month are lower than 10, in which case it's missing the left-padding zero. Luckily, for the purpose I was using it, it worked anyway. Because OF COURSE there's no included function to left-pad a string. You have to do it by addition. Or, of course, add a 3rd party function that's out there, in the internet, somewhere.
I used to manage a large number of QMail installations. And because Qmail was ... weirdly licensed, I wrote a set of plugins that ran on top of a patch called Qmail-SPP. I pretty much stopped doing that years ago because life took me in other directions, and forgot all about it.
That collection is called ra-plugins and I had not touched it since late 2008.
And today... I got a patch with two whole plugins to add to it so that it makes Qmail handle email addresses more like Gmail does (aliases using user+foo and making user.foo the same as userfoo).
So, I got them, added them, fixed a few simple building issues, updated the libsmtp it uses internally for one of the plugins to a later version, and there it stays, perhaps not to be touched until 2018.
Over a year ago (time flies!) I posted something about a project called Alva. Let me quote myself:
Alva is almost the opposite of Nikola. If Nikola is about making static sites, Alva is a dynamic site. However, as Hegel suggests, from the thesis and the antithesis comes the synthesis.
So, Alva is about dynamically creating static sites. If you want to have Nikola in your server instead of in your own computer, and have the convenience of an online tool, that's the niche Alva tries to fill.
So, you would install Alva, and use it like any other web-based blogging tool. Yet, behind the scenes, you would have Nikola, and all the performance and security benefits of static sites.
And maybe someday, I (or someone) will put up a multi-user version of Alva, and you will be able to get hosted blogs, knowing all the data is yours and you can leave anytime and do your own thing.
The approach I was taking at the time proved to be unsuccessful, and there were a few other failures along the way. Of course, the problem was in how I was approaching the task. So I did the right thing, and learned how to do it "right".
Still not usable, still not hosted anywhere, but already semi-functional: Alva lives now
There's a lot of work still to be done. But I now know how to do it. To prevent the usual arguments, here is a little explanation of motivation, tooling, etc.
I want a way to host blogs very cheaply. How cheaply? I want at least 1000 reasonably active users in a $5 VPS. That would make Alva a reasonable alternative to hosted multi-user wordpress, which means it would be a reasonable solution (if setup is easy enough) for small-to-medium organizations which don't want to setup expensive infrastructure yet want to own their data (think schools, small businesses, FLOSS projects, etc.) I also want to provide that service, for free. Which is why another reason I want it to be super cheap.
How does Alva help provide this super-cheap blog hosting?
It needs to scale following the number of edits not views.
If it gets too busy with edits, changes take longer to appear, but the site itself doesn't get any slower.
Editing and serving can be provided by separate services, so I can use some super-fast static file server and a super-configurable WSGI deployment.
Individual pages can be heavily optimized so that they download fast
One of the guiding principles here is that to deliver this sort of thing, in my spare time, the development process needs to be stingy with the most limited resource: me. I can't spend a lot of me here. I need to be careful and not over-promise.
So, whenever there was a 3rd-party tool that saves a significant amount of time, that's what I am using.
Because it has a much stronger 3rd-party toolset than Flask or any micro-framework. For example, the Flask equivalent of django-allauth broke my will to live. Because the admin interface means I can start adding data to see if it makes sense before I write all the required views.
Because I don't want you to have to create accounts here unless you want to, this provides (optional) social login and registration. This was easy to setup and works almsost out-of-the-box
- Bootstrap and Django-bootstrap-toolkit
Nikola is already heavily invested in bootstrap, so it just made sense to go further down that road. I understand bootstrap, and django-boostrap-toolkit is easy enough (although I can't make their datepicker work)
Because fighting is boring.
Because Django's mechanisms to find templates and static files are many and confuse me.
- Redis + RQ + django-rq
It's crucial for the whole approach to use job queues in order to detach the rendering from the actual Django app. This combination makes job dispatching ridiculously easy, setup is trivial (install everything, start redis, a few lines of config,
./manage.py rqworkerand off you go) and they provide a django admin page where I can see the failed jobs, which is awesome.
Because it's easy enough, and allows me some freedom exploring data organization in my models without committing to it forever or recreating databases with test data all the time.
I will probably serve the generated sites via gatling just like my current sites because it has the simplest named domain configuration possible, it's fast and very light in resource usage.
A cool, simple editor with live previews that supports almost every markup. Not WYSIWYG or even WYSIWYM so possibly I will have to add an alternative. I started using django-markitup but it's not a good idea (it uses a old version of markitup which requires JQuery < 1.9) and am in the process of just using Markitup manually.
So, feel free to give Alva a try and/or give me a hand, comments welcome.