APT Pupil: your own repository

Intro

In a previous article, I explained how APT can improve your Fedora (or similar) system administration experience. In yet another, I explained how to create your own RPMs without much trouble.

Well, you can surely see how there is a need for this article, explaining how you can use your own RPMs via APT, right?

So, based on the principle that you can't have too much of a good thing, here it is: a (hopefully) simple guide to building your own APT-RPM repository.

There is already at least one decent such guide: It's good!. I will go into a little more detail, though.

Target audience

Will you get anything from this article? I think you will if:

You are a sysadmin, or play one for your friends and family.
You manage more than one or two Linux systems.
You use a RPM-based system like Red Hat, Fedora, Conectiva ( maybe Suse or Mandrake)
You like to use APT
You install some software from outside the usual APT repositories, either custom or self-packaged.
You are not familiar with repository management.

This is not for you (except by chance), if you fit any of the following criteria:

You use a source-based distro
You use a Debian-based distro (the setup of the repo is slightly different)
You manage a single system
You don't like APT
You don't build your own software or packages

Uniformity is one of the basic principles of software management. If you want to lower the work involved in managing multiple systems, you should try to install software as similar as possible in all of them.

To do that, a decent system of software delivery is necessary, so installing the new versions or new packages in all systems is not a hard chore. I like APT. But, like all repository-based systems, it has the weakness that, if you install something that's not in your repos, you are outside the system.

Sure, you can just opy the RPMs to each server and install it, but that's what APT is for. The simplest solution is just building your own repo, and handle it like the rest of the stuff you install.

And the most important reason: it's easy. So easy that even if it gives only a small benefit, it's still worth it.

Ingredients

You will need several things, including:

A web server
Some apt tools for repository management
Some RPMs

The web server

We'll build a HTTP-based repository. The reason is simple: it's usually easier and safer to build a HTTP server than a FTP server.

Please notice that I act as if you own the web server. If you don't, just do all this in your own box, and then upload all the repository into the webserver. That works just fine, and has the added benefit that the web server is Somebody Else's Problem (TM pending).

If you know Apache, and are happy with our Apache setup, then just use it. However, using such a complex piece of software just for a repo is extreme overkill.

If you are starting from scratch, and you expect to use the server only for APT, and you don't expect it to have high traffic, then you can do just as well using a very simple HTTP server.

These features are useful, however:

Ranges: This allows the server to handle continuation of downloads, and may save lots of time.
Pipelining: Again, this may help performance.
Download rate limiting: So it doesn't kill you if a million guys link to it unexpectedly.
A good security history
Actively maintained
Simple configuration

And, most important: Make it simple. I am a fetichist of simple software. All these features are NOT useful for this job:

PHP support
CGI support
Virtual hosts

All you need to handle with this webserver is fetching files. Maybe a directory listing, if you enter manually. Almost every webserver is massive overkill.

The bad news is, there are hundreds of web servers you can use, and I know very few of them.

What do I recommend? Well, I use Apache myself ;-) But here are a few possible cadidates, which I don't endorse in any way.

Thy: Maybe too featureful (and little known?).
thttpd: Looks nice.

And many others, like Boa. Even KDE's KPF can work in a hurry.

However, this article is not about creating a website, so the actual task of publishing a site is left as an exercise.

The APT repository tools

They came with APT for me, so, if you are using APT in your system, you should have them too.

To check it, see if you have genbasedir installed, like this:

[ralsina@roberto ralsina]\$ which genbasedir
/usr/bin/genbasedir

If instead of giving you a path it gives an error, then you don't have it.

Some RPMs

Almost anything will do. You may, for example, just use all the RPMs from your distro's CD set. That way, you can install them without searching on what CD they are ;-)

Configuring the repository

A APT repository is a directory structure containing the RPMs, plus some extra database files created using the APT tools.

The directory structure

Split your RPMs in a natural way, if needed. For example, you may have tested and untested RPMs, or custom (the ones you made) and os (the ones that came with the distro). Whatever split you make, makes no difference, really. I will use stable and unstable in the example.

If you use different distributions or architectures, that matters too. I use only Fedora and only i386, but for the example, I will pretend to use Red Hat 9 (rh9) and Fedora Core 2 (fc2).

Now, go to a directory you share through your webserver and create something like this:

mkdir -p rh9/RPMS.stable
mkdir -p rh9/RPMS.unstable
mkdir -p fc2/RPMS.stable
mkdir -p fc2/RPMS.unstable
mkdir -p rh9/SRPMS.stable
mkdir -p rh9/SRPMS.unstable
mkdir -p fc2/SRPMS.stable
mkdir -p fc2/SRPMS.unstable

As you can see: a directory for each architecture-distro you support, and inside it, a RPMS and SRPMS folder for each set of RPMS. you want to distribute.

If you are not distributing SRPM files (why not?) you can skip those.

Now, put your .rpm files in the RPMS directories, your .src.rpm files in the SRPMS directories.

Generating the database files

This is what makes this site a APT repo instead of just a bunch of RPMs on a page.

genbasedir --progress --flat --bz2only /whatever/rh9
genbasedir --progress --flat --bz2only /whatever/fc2

Just to be sure: the directory you pass as argument is the one where the RPMS.xxx you created before is. After you run genbasedir, you will have a base/ directory there.

You need to repeat this whenever you add or remove files to your repository.

Using your repository

First, read this other article ;-) Specifically, this part.

In the example setup I'm doing, assuming my webserver is at http://myserver.com and that the rh9 and fc2 directories I created can be seen as http://myserver.com/apt/rh9 and http://myserver.com/apt/fc2, the sources.list file should contain this:

#If you are using Fedora Core 2
rpm http://myserver.com apt/fc2 stable unstable




#If you are using Red Hat 9
rpm http://myserver.com apt/rh9 stable unstable

Notice how the RPMs are in places like http://myserver.com/apt/rh9/RPMS.stable and the sources line is simply a chopped up version of the URL.

Of course you can choose to use only stable (or only unstable ;-).

Coming up eventually

While APT is all fine and dandy for what it does, in order to do really good package management, it's not enough to just do apt-get update and apt-get upgrade every once in a while.

You need to know how to choose a specific version of a package, and how to prevent unexpected upgrades that can break down your configuration.

So, if I ever have the time and energy, I should write another article covering policy, pin, and other less-popular pieces of APT.

But that's another story.

Final Words

Hope you enjoyed the article. If you did, there's more in the Articles section of the site.

If you have any doubt or question about any of this, just post a comment or email me. The best thing is probably to post a comment and email me telling me you posted it

Ralsina.Me — Roberto Alsina's website