I was looking for a basic tutorial to configure a server with puppet but I couldn’t really find one (leave a comment if you know of a good one). At the time I was kind of surprised I couldn’t find one but now I think I see why. The process is frustrating and doesn’t necessarily translate well to people with other needs. This post and the accompanying demo project are my attempt at writing the tutorial that I was looking for. Which might not be the tutorial you are looking for.
I wanted a simple repeatable way to set up a rails project and then deploy it. Figuring out how to do this reliably wasn’t as simple as I hoped it would be so I hope this will be simpler for others in the future.
The reason I wanted this was because it makes things easier and less fragile. If the worst happened to my development environment or, perish the thought, production then I could could be up and running a few minutes (ok more like 20, it can be slow) after running one command. Of course if this did have to happen to production this method wouldn’t restore backups but eventually I’d like to have something like that implemented. Things are also less fragile this way because you can simply set up staging and development environments that aren’t reliant on the random assortment of things you have installed. This also lets you keep what is installed and what files are on your server under version control.
This also makes things like setting up staging servers easier. And people like me are much more likely to do things if they are easier.
The main tools I chose to use for this are: Puppet, Capistrano, and Vagrant. I chose Puppet because I like that it is declarative and I’ve never really used any configuration management system that wasn’t shell scripts so I just picked one that seemed fairly popular. I used Capistrano because it seems to be the standard for deploying rails projects and so it already does almost of all of the work required for deploying a rails app. I used Vagrant because it makes dealing with VMs easy and I’m not aware of anything similar.
With Puppet I also used the librarian-puppet gem which is sort of a bundler equivalent for puppet modules. I’m using this because when I was trying to get this to work I had a lot of problems with installing puppet module dependencies and using that helped me sort them out.
I’m using masterless puppet for this because none of my projects have requirements that one small server can’t fulfill. If you need to do something that requires multiple servers this project might be a useful starting point but it will need a bunch of fiddling before it will start to be useful.
Using the example project
To run the example you need to have an ssh public key from
to copy to
If you don’t have any public keys and don’t know how to generate one
github has a nice tutorial.
I’m also assuming you alread have vagrant
and virtual box installed.
Then all you have to do are run the following commands:
git clone https://github.com/beardo/puppet-capistrano-demo cp ~/.ssh/your_key.pub puppet-capistrano/puppet/files/all/ssh_authorized_keys cd puppet-capistrano-demo/puppet vagrant up cd .. # the password it's about to ask you for is vagrant cap vagrant cold_start
Then go to 192.168.123.21/hello_world
and if you see something like this:
Then it worked!
If not I’m not sure what went wrong.
Maybe leave a comment with the last few lines of the
file on your vagrant machine and I’ll try and help.
To get that file run:
vagrant ssh sudo tail /var/log/nginx/error.log
Adapting the project for your own use
Pretty much everything in the
puppet/ directory is required for this to work.
You don’t really need my vimrc file because it’s mostly there as an example.
If you remove it be sure to remove the part in
that refers to it or you’ll get errors during your puppet install.
Because I haven’t figured out the hiera and secrets stuff yet
(more on this later) the
deploy user created by puppet
and used by
capistrano doesn’t have a password.
Ok it also doesn’t have a password because
I didn’t like having to type one in my testing.
Regardless you have to either add a password in
puppet/files/all/ssh_authorized_keys to add a public key like is
puppet/files/all is uploaded to your server
along with whatever is is
So if you’re deploying to vagrant then everything in
puppet/files/vagrant is sent
or if you’re deploying to staging then everything in
be sent and so on.
The reason I set it up this way is so that I can have different versions of files on
Mostly I use this for my nginx configuration.
In the vagrant stage there isn’t a hostname but in production there is
and in staging I block all traffic that isn’t from me
which I obviously don’t want to do in production.
puppet/fileserver.conf file is necessary for using masterless puppet.
Basically what it does is tell masterless puppet to look for source files
in the root of
which is where capistrano uploads them to.
You can change this and I’m thinking I might change it to
because in retrospect I don’t really see a reason to keep them persisted.
In “normal” non masterless puppet these files would probably be somewhere on
the puppet master and copied out to the nodes from there but again that’s
overkill for my stuff.
puppet/Puppetfile.lock are like
Gemfile.lock but for puppet modules instead of ruby gems.
This simplifies installation quite a bit.
For more information on how to use these see
librarian-puppet on github.
manifests/init.pp is meat of our puppet install.
It installs all of the packages we need and sets up
all the users and files for us.
Honestly it should probably be broken up into smaller files
and simplifying any future moves to needing multiple servers.
But I’m still trying to figure this puppet thing out
and I don’t know the best way to do that right now
so for now it’s one monolithic file.
Currently it installs and configures:
- our application’s database
- nginx (with passenger)
- redis (which we don’t use but I left in there because evenutally)
- a specific ruby version for our application (2.1.5)
- also bundler
- our deploy user
- some utility packages (htop and vim)
I tried to include comments with more information and my reasoning for why throughout the file.
To run capistrano for this project you need
Capfile specifies what all capistrano modules you’re using and where your
capistrano tasks are located.
config/deploy.rb is your where you do your global configuration for capistrano.
You’ll probably want to change this file
unless you really only want to run versions of this demo.
To get started all you really need to change are the following two lines
(it looks like more because I’m long winded):
# change this line to whatever your project name is # you also should also update puppet/manifests/init.pp # and puppet/files/vagrant/puppet-capistrano-nginx # because they both refer to the location set by this variable set :application, 'puppet-capistrano-demo' # update this with a link to your git repo or you'll just # keep getting versions of my demo project set :repo_url, 'firstname.lastname@example.org:beardo/puppet-capistrano-demo'
The files in the
config/deploy/ directory setup what capistrano calls stages.
In these files we define roles
and give the user and address of those roles.
The default (and required) roles for capistrano are:
Since I only need one server these are all the same for me.
I also defined a separate role called
This role is only used for setting up our server to use puppet.
and actually running puppet on our file.
This role has a different user than the others.
That’s because this role needs to run before the
deploy user exists
and because this role needs to have sudo which deploy does not need.
no_release: true makes sure that this role is
ignored in normal
cap stage deploy commands because it isn’t needed there
and so it can use different authentication methods than the
define three capistrano tasks:
cap vagrant puppet:prepare) This task sets things up so you can run puppet on your stage. It does an
apt-get update, installs ruby, puppet, and the librarian-puppet gem.
cap vagrant puppet:install) This command actually runs puppet on your stage. It uploads everything you need to run puppet and then runs puppet. Afterwards your server is how you defined it to be.
cap vagrant cold_start) This task is basically just a wrapper task. It runs
deploy. Using this one command you can go from a fresh install of ubuntu 14.04 to having everything installed and set up for your app in one command.
If you change things in your puppet manifest
or files you need uploaded you can
cap stage puppet:install
and puppet will update your stage.
If only update your app code and want to deploy those changes
without running puppet again
you can run
cap stage deploy and capistrano will only
update your application.
Issues and future plans
The main issues I have (I’m sure there are plenty of others) with this implementation are:
- The passwords aren’t secure and you shouldn’t use them.
I mean other than the fact that they’re “password”.
They’re just sitting around in plaintext.
- The solution to this is to use Heria and possibly something like heira-gpg or heira-eyaml to keep your secrets out of plain text. Like I said earlier I haven’t gotten far enough along yet to work all of this out but I’ll update this post once I have. Hopefully that will be fairly soon but it probably won’t be that soon.
- No way to restore from backup. One of the reasons I wanted to do this was to make disaster recovery simpler in case something happened. Being able to recover from backups automatically would really with that. Plus it would make me much more likely to test that the backups worked. And as we all know having untested backups is basically like not having backups at all. This will probably take a lot longer than the heira stuff because it’ll be a while before this project will need backups.
- It would also be nice to be able to programmatically create the VPS my application is running on using something like the fog gem but that’s not a huge priority for me right now.
- At some point having the option to have multiple servers in this setup would be nice. At the very least for tutorial purposes but it’ll probably be a while before I need that.