Photo of David Winter

david winter

Introduction to Puppet

Have you ever had that déjà vu feeling when setting up a new server? That you’ve done it all before, and that it’s really tedious to have to do it all over again? You’d rather be able to just hit a button to deploy your app and start showing it off to the world.

It’s exactly what happened to me, and at first I thought I’d just write a bash script with all the various steps that I had started noting down in a plain text file. Luckily, at the point I was considering this, I saw the release of a new project called Vagrant. It’s a tool that allows you to run a virtual machine locally, with a helpful shared directory to your files on your local (host) machine. It allows you to setup a self contained development environment without messing around with your system.

Reading through the documentation, I saw it mention Puppet and Chef which both appeared to be some sort of witchcraft that allowed you to automatically configure and setup these virtualised servers. Bingo. Easy server configuration management.

You don’t have to use Vagrant to use Puppet. It’s just a really easy way to test it out locally. For your production environments you’ll still use the same puppet manifests, but you’ll manage the running of those manifests a little different. I’ll be covering the Vagrant approach for this blog post.

What do I want to demonstrate? Just a quick demo of how to get Puppet to install Nginx and php-fpm on a server, serving a simple phpinfo() file. It should be enough to make you thirtsy for more.

Some quick steps for setup

You’ll need to have Virtual Box installed with the command line tools option. Then install the Vagrant gem on your computer with a simple

gem install vagrant

Now create a new directory on your machine where you’re going to work on this project.

mkdir testing-vagrant
cd testing-vagrant/

All Vagrant projects use a ‘box’ as a starting point for your project. It’s an virtual disk image of an OS installation that has all the dependencies to interact with Vagrant, and to run puppet manifests. Now, the one on the Vagrant website is Ubuntu 10.04 Lucid Lynx, and without some tinkering, it’s not as easy to get the latest versions of packages. So instead, I created my own Vagrant box for Ubuntu 11.10 Oneiric Ocelot using the open source tool VeeWee. It’s hosted on my public Dropbox folder, and to add this box to your vagrant setup, run the following:

vagrant box add oneiric32 http://dl.dropbox.com/u/11342885/oneiric32.box

Once this has been downloaded, you need to initialise your Vagrant project with this box:

vagrant init oneiric32

This will create a file named Vagrantfile in your project directory. This file has a bunch of different configuration options commented out. It’s a really helpful file to read through.

One configuration option you’ll need to set is the port forwarding of the webserver. Ensure this line is present:

config.vm.forward_port 80, 8080

This will forward on the port 8080 of your local machine to port 80 on the vagrant virtualbox. If you don’t have any other webservers running on your local machine, feel free to point this to port 80 directly.

Also, ensure these lines are present too:

config.vm.provision :puppet, :module_path => "private/puppet/modules" do |puppet|
  puppet.manifests_path = "private/puppet/manifests"
  puppet.manifest_file  = "base.pp"
end

This is the Puppet configuration. It tells Vagrant where the modules and manifests directories are, and the base manifest file.

And finally, create these directories in your project for storing the different Puppet configuration files we need:

mkdir -p private/puppet/{manifests,modules/nginx/files}

Playing with Puppet

To understand how to use Puppet, you need to think of a Puppet manifest as a file that defines the different resources your server requires. Resources in Puppet speak are things like users, groups, packages, commands, services and files–to name a few. We define these resources in a manifests file which you’ll place in private/puppet/manifests/base.pp.

Now the first thing we want to do with the base box is ensure that everything is up-to-date. The first resource we’ll define is an executable command to run. Basically an apt-get update. In your base.pp file add the following:

exec { 'apt-get update':
	command => '/usr/bin/apt-get update',
}

All resources follow this same structure:

resource_type { 'title':
	options => value,
}

The title is quite an important value, as it is often used as a default option value too. For example, with the user resource, the title you specify will be the name of the user to create. This can be overriden if you need to define a descriptive title that isn’t the name of the user you want to create. For the user resource, you’d override this with the name option. For example, the following would both create the same user:

user { 'david':
	ensure => present,
}

user { 'my descriptive title for my username':
	name   => 'david',
	ensure => present,
}

Note: Due to an issue with VeeWee not creating a required user group on the Oneiric box, you need to also tell Puppet to create the puppet group. This can be done with the following. Not doing this will result in errors:

group { 'puppet':
	ensure => present,
}

Back to our example. We now have a command resource that will run apt-get update. Once this has been run, we want to install the packages nginx and php-fpm. We want to ensure though that apt-get update has run beforehand. Now, with Puppet, it’s important to note that the manifest files are not run through sequentially as you might expect. Each time they are run, they can happen in a different order. You need to explicity set dependencies and requirements.

To install the nginx package, we’ll add the following:

package { 'nginx':
	ensure => present,
	require => Exec['apt-get update'],
}

A few things to note here. The package to install is being derived from the resource title. It defaults to this, or you can specify the option with the option key name. We tell puppet to ensure that it’s present on the system. We also say that this resource requires that the executable command apt-get update has already been applied.

When you reference other resources in a manifest file, the resource type has the first letter capitalised. Then you specify the title of the resource you’re referencing. This is where resource titles are used.

We’ll also do the same for the php-fpm package:

package { 'php5-fpm':
	ensure => present,
	require => Exec['apt-get update'],
}

Each of these packages come with a service that runs in the background, that we want to ensure is running all the time. We can make Puppet check this is the case with the following:

service { 'nginx':
	ensure => running,
	require => Package['nginx'],
}

service { 'php5-fpm':
	ensure => running,
	require => Package['php5-fpm'],
}

Notice the require dependencies? Whenever Puppet runs it will check that these are running, and if they’re not, for whatever reason, it will start them.

Ready to see some magic? Save the file and run vagrant up. Vagrant will boot up the virtual machine, and run the Puppet manifest on it. Once it’s finished visit http://localhost:8080 and you should see:

Welcome to nginx!

You’ve not had to ssh into the machine once or run anything on the command line of that VM.

Now in order to get PHP setup and running with nginx, we need to modify some of the nginx config files. We need to do the following:

  1. Disable the default nginx virtual host.
  2. Copy over a new virtual host for our vagrant setup that forwards on PHP requests to php-fpm.
  3. Symbolically link the virtual host config file to the sites-enabled directory.

In the private/puppet/modules/nginx/files directory we created, add the following in a file called vagrant:

server {
	listen 80;
	server_name _;
	root /vagrant;
	index index.php;

	location / {
		try_files $uri /index.php;
	}

	location ~ \.php$ {
		fastcgi_pass 127.0.0.1:9000;
		fastcgi_index index.php;
		include fastcgi_params;
	}
}

This is a standard nginx configuration file for a virtual host. It will use /vagrant on the virtual machine as the document root. This is a shared directory that Vagrant sets up to link to your project directory on your local machine.

Saving that file, we can now open up the base.pp file again and add a few more resources. The first is to copy over this virtual host to the correct location:

file { 'vagrant-nginx':
	path => '/etc/nginx/sites-available/vagrant',
	ensure => file,
	require => Package['nginx'],
	source => 'puppet:///modules/nginx/vagrant',
}

Now lets disable the default virtual host the nginx package provides:

file { 'default-nginx-disable':
	path => '/etc/nginx/sites-enabled/default',
	ensure => absent,
	require => Package['nginx'],
}

And finally, enable our new vagrant virtual host:

file { 'vagrant-nginx-enable':
	path => '/etc/nginx/sites-enabled/vagrant',
	target => '/etc/nginx/sites-available/vagrant',
	ensure => link,
	notify => Service['nginx'],
	require => [
		File['vagrant-nginx'],
		File['default-nginx-disable'],
	],
}

Save the file and run vagrant reload. This tells vagrant to run the puppet provisioner again and ensure that the virtual machine matches the resources specified in the manifests file. We’ve just added three new ones, so it will action these.

Ready to test it has all worked as expected? Create an index.php file in your project directory, the same location where your Vagrantfile is. In there just put a simple:

<?php phpinfo();

Save the file. Visit http://localhost:8080 again and you should see the standard phpinfo page being served over php-fpm via nginx.

That’s it for this introduction. Hopefully it’s enough to get you excited. For me, it means being able to keep a manifest file up-to-date, and if I ever change my VPS to a different provider and want to get it up and running within a few minutes rather than hours, I can apply it with Puppet. No more keeping a log of the many steps I would have to manually run over ssh as the alternative.

If you want to know more, here are some great resources: