Happy New Year

Happy new year everyone! I hope you could spend a nice time with your family and friends.

I'm very sorry, that the third part took me so long. The end of the year is always a tough time, but this year was especially busy. We had a lot work over at dajool and I needed to take some time off during the holidays to relax a bit.

Deployment

In part 2 we looked into the details of the package generation. Now I'd like to give you an idea of how to deploy the packages and even how to automate the deployment.

The Easy Way With scp… For Testing

After building the deb package you will want to deploy the package. For a simple test, you can just scp the deb package to the server and directly install the package with dpkg -i my_package.deb. However dpkg might complain about missing dependencies. Note that this won't happen, if you are able to install the package with apt-get - like when you are installing from your own repository. Just type apt-get install -f to install the missing dependencies.

Full Blown Deployment With reprepro

Manually copying deb file to multiple servers is not very convenient and no time saver at all. This is where reprepro comes into play. reprepro is a Debian package repository producer. This means reprepro just builds the folder and file structure needed for a repository, which might look like this:

.
├── conf
│   ├── distributions
│   └── options
├── db
│   ├── checksums.db
│   ├── contents.cache.db
│   ├── packages.db
│   ├── references.db
│   ├── release.caches.db
│   └── version
├── dists
│   ├── precise
│   ├── quantal
│   └── raring
├── incoming
├── keys
│   └── public1.key
└── pool
    ├── contrib
    └── main

An installation and configuration howto can be found in this wiki article from debian.org. It basically tells you how the configs should look like and how the GPG-key can be generated. Generating a GPG-key is highly recommended if you are really going to use this setup. Otherwise apt-get won't be able to verify the origin and authenticity of the deb file and will complain. If you prefer Nginx over Apache you can use this very simple, uncomplete but sufficient sample config.

Your distribution file might look like this:

Origin: neverland
Label: neverland - precise
Suite: stable
Codename: precise
Architectures: i386 amd64 powerpc
SignWith: 7B8F4ACB
Components: main non-free contrib
Description: Ubuntu repository for precise packages from neverland.

The 'Codename' should always be conform to the codename used in the sources.list - otherwise this will be a nice source of errors for anyone else but you. If you'd like to add more distributions, just copy this section and make the distribution specific modifications.

After reprepro knows which distributions and architectures you'll want to host, you can add new deb packages to your repository like this: reprepro -b <path_to_repo> includedeb <codename> <path_to_deb>

The package is immediately available and can be installed with apt-get update && apt-get install my_django_app – at least if you added the repository to your /etc/apt/sources.list. To add this repository to your sources.list you'll just have to append the line deb http://my_neverland_domain.com precise main.

That's it. Now everyone on the planet can apt-get install packages from your server. apt-get also supports https and basic authentication - just in case you want to host more private deb files. You'll just have to adjust your Apache or Nginx config.

RPM users can use createrepo explained in this blog post.

Automated Deployment With SaltStack

If you are not familiar with SaltStack; It's an infrastructure management tool to orchestrate your servers (aka minions) from your master. Salt can do everything: You can execute commands on the minions, do things periodically, distribute config files or install packages. You should especially take a look at the grains. With the information SaltStack collects from the system - the grains - you are able to target minions from a certain group/role or location, with a specific OS, but also minions with e.g. four CPU or just the minions with a specific GPU if you are doing some CUDA number crunching.

A very nice installation instruction can be found in the excellent documentation.

Now you can do multiple things. Let's assume that you are using Jenkins as a build server. You trigger builds either from an incoming hook in your central mercurial repository or just let Jenkins check the mercurial repository periodically for changes. Git uses a very similar system. The deb files (artifacts called in Jenkins) then get pushed into reprepro by the salt master, which updates the package on your test server afterwards for QA.
It would also be possible to not use Jenkins at all and manage the builds from the salt master. But then you would have to write some more glue code. You could also use a specific branch in your mercurial repository or tags for production builds. The possibilities are endless.

Nice, isn't it? Since this topic ist so manifold and interesting I would like to start an experiment here. I've created a mailing list on librelist.com. What are your thoughts on deployment with packages? Do you have questions, annotations or do you just want to discuss a part of the setup? Just send an e-mail to happyadmin@librelist.com and you'll be subscribed to the list. Any e-mail send afterwards will be posted to the list.