david winter

Keeping Claude Code MCP credentials out of git with 1Password

2026-07-08T00:00:00+01:00

I commit parts of my Claude Code setup to git so it follows me between machines and stays version controlled. The trouble is that MCP servers often need credentials, an API token here, a bearer token there, and those absolutely cannot go into a file that’s tracked in a repo. I already keep everything in 1Password and use its CLI daily, so the answer was to have Claude Code pull the tokens from there at the moment it needs them, rather than store them anywhere on disk.

The result is a .mcp.json I’m happy to commit. It describes every server, but holds no secrets. Each one that needs a credential resolves it at launch through a helper command that reads from 1Password, so the token only ever exists in memory while the server is running.

How the credential gets injected

Claude Code supports a headersHelper field on an HTTP MCP server. Instead of a static header value, you give it a command, and Claude Code runs that command when it starts the server and merges the output into the request headers. The command has to print a single JSON object mapping header names to values.

That’s the hook. The command reads the secret from 1Password with op read and the op://vault/item/field reference format, so the reference (which is safe to commit) lives in the config, and the actual value is fetched on demand.

Note: you need the 1Password desktop app installed, with CLI integration enabled under Settings > Developer > “Integrate with 1Password CLI”. I’m frequently between a mixture of WSL, Linux and macOS, but on WSL, the CLI is the Windows binary invoked as op.exe; on macOS or Linux it’s just op. In future, I’ll create an alias within my Fish shell so that op is available on all platforms!

Adding a server the right way

The important detail is to add servers with claude mcp add-json, not the plain claude mcp add. The plain command’s --header flag only stores static values, which would defeat the whole point. The JSON variant takes a raw blob, so it’s the only path that can carry the dynamic headersHelper field.

Store the credential in 1Password first, note its op://vault/item/field reference, then feed the template to add-json:

claude mcp add-json --scope project <server-name> '{
    "type": "http",
    "url": "<mcp-endpoint-url>",
    "headersHelper": "printf '"'"'{\"<Header-Name>\": \"%s\"}'"'"' \"$(op.exe read '"'"'op://<Vault>/<Item>/<field>'"'"')\""
  }'

Remember, change op.exe to just op on macOS or Linux.

Restart Claude Code afterwards and confirm the server connects. The quoting looks fierce, but you’re only ever changing the header name, the URL, and the op:// reference.

A single secret

Most servers need one bearer token in an Authorization header. Here’s GitHub, pinned to read-only tools with the /readonly on the URL:

claude mcp add-json --scope project github '{
    "type": "http",
    "url": "https://api.githubcopilot.com/mcp/readonly",
    "headersHelper": "printf '"'"'{\"Authorization\": \"Bearer %s\"}'"'"' \"$(op.exe read '"'"'op://Employee/GitHub token/credential'"'"')\""
  }'

More than one secret

When a server needs two credentials, send them as two headers. Extend the printf format string with another %s and add one op.exe read per placeholder, in order. Datadog wants an API key and an application key:

claude mcp add-json --scope project datadog '{
    "type": "http",
    "url": "https://mcp.datadoghq.eu/api/unstable/mcp-server/mcp",
    "headersHelper": "printf '"'"'{\"DD_API_KEY\": \"%s\", \"DD_APPLICATION_KEY\": \"%s\"}'"'"' \"$(op.exe read '"'"'op://Employee/Datadog API key token/credential'"'"')\" \"$(op.exe read '"'"'op://Employee/Datadog MCP token/credential'"'"')\""
  }'

With this in place my Claude Code config is fully committable. The tokens stay in 1Password, unlocked by the desktop app I’ve already authenticated, and they’re pulled into memory only when a server starts. Nothing sensitive now ever touches the repo.

Making Firefox open new tabs where I expect

2026-06-22T00:00:00+01:00

I keep a lot of tabs open. Probably too many. I also lean on pinned tabs for the handful of things I always have running, like email and my calendar. The trouble is what happens when I open a new tab: instead of landing at the far right where I go looking for it, it would often appear at the far left, wedged in right after my pinned tabs. With a busy tab strip that’s just enough friction to be annoying every single time.

After some digging in about:config, I’ve settled on three settings that get the behaviour I want:

browser.tabs.insertAfterCurrent              false
browser.tabs.insertAfterCurrentExceptPinned  true
browser.tabs.insertRelatedAfterCurrent       true

To change them, open about:config, search for each name in turn, and set the value. Two of these are actually Firefox’s defaults; the third is the one that fixes my real gripe. It’s worth walking through what each one does, because they only make sense together.

Opening a blank tab

When I open a fresh, empty tab with Cmd+T or the + button, I want it at the far right, on the end of the strip. That’s browser.tabs.insertAfterCurrent left at its default of false. With dozens of tabs open, a new blank tab belongs at the end where there’s room, not shoved into the middle next to whatever I happened to be looking at.

Opening a link from a normal page

When I open a link in a new tab, by middle-clicking or using Open Link in New Tab, it opens immediately to the right of the tab I opened it from. That’s browser.tabs.insertRelatedAfterCurrent set to true, also a default.

This one I genuinely want. If I’m reading an article and open three links from it, they line up right next to that article rather than scattering to the far end. The new tabs are clearly related to where they came from, and when I’m done I can close the little cluster without hunting around.

Opening a link from a pinned tab

Here’s the setting that does the real work: browser.tabs.insertAfterCurrentExceptPinned, set to true.

A pinned tab counts as the current tab, so the “related after current” rule above would happily drop a new tab right after my pinned tabs, at the far left. That was the exact behaviour driving me up the wall. This setting carves out an exception: when the current tab is pinned, send the new tab to the far right end instead of crowding it in beside the pinned ones.

The result is that my pinned tabs stay a tidy, fixed cluster on the left, and anything I open from them shows up on the right where I’d naturally look. Best of both worlds!

The catch: these don’t sync

There’s one real frustration. I use Firefox Sync to keep most of my setup consistent across devices, extensions especially, and it does a great job. But about:config preferences like these aren’t part of what it syncs. Set them up on one machine and you’re starting from scratch on the next.

So every time I set up Firefox somewhere new, I’m back to hunting through about:config from memory. That’s exactly the kind of repetitive, easy-to-forget setup my dotfiles repo exists to handle, and I’m exploring whether I can manage these preferences from there instead, most likely through a user.js file dropped into the profile. I’ll write that up separately if I land on something that works.

For now though, my tabs finally open where I expect them to. Small win, daily payoff.

RustDesk direct connections over Tailscale

2026-06-06T00:00:00+01:00

I use RustDesk to help family with their computers from a distance. Out of the box it routes through the public RustDesk server, which is the path of least resistance when you’re getting started. But, that changed recently; a login is now required for the public server, and that’s because the public server was only ever meant for testing and demonstration apparently, and because of growing botnet and scam abuse.

I didn’t want to have to self-host a relay server for the handful of machines I connect to. All of those machines are already on my Tailscale network, so they can already reach each other. That makes the public server redundant. I can point RustDesk straight at a machine’s Tailscale IP and skip the middleman!

Enabling direct IP access

On the machine you want to control, open RustDesk and click the three-dot menu next to its ID, then go to Settings. Under the Security section, click to unlock it, and tick Enable direct IP access.

This is what lets a RustDesk client connect to this machine by its IP address rather than going via the relay.

Setting a permanent password

While you’re still in the Security section, scroll up to the Password area and set a permanent password. Without one, RustDesk generates a temporary password that rotates, which is no good when you want to connect again later without being sat in front of the machine.

I store the password in 1Password so it’s there whenever I need it. Note: treat this like any other credential, since anyone with the password and network access to the machine can connect.

Finding the Tailscale IP

Each machine on your Tailscale network has its own stable IP in the 100.x.x.x range. You can grab it from the Tailscale client in the menu bar or system tray, or from the machines list in the admin console.

That IP stays the same as long as the device is part of your network, so it’s worth noting down alongside the password.

Connecting

On the machine you’re connecting from, paste the Tailscale IP into the Control Remote Desktop field and connect. Enter the permanent password when prompted, and you’re in.

RustDesk will warn that this is a direct and unencrypted connection. That warning is about RustDesk’s own transport, but the traffic is already running inside Tailscale’s encrypted tunnel, so it’s safe to proceed.

That’s the whole setup. As long as both machines are on Tailscale, I connect directly, with no shared server and no login in the way! Problem solved.

Post-deployment triggers for PaaS providers

2023-03-14T00:00:00+00:00

For really simple projects, PaaS providers such as Fly.io and render.com offer a pain-free way to get an app or static site up and running in minutes. Sometimes you may want to trigger some follow up actions after a deployment, for example, a Slack notification to notify your team that a new release is live, or some smoke tests.

The above two providers are quite different in how they handle deployments. Fly.io lets you run a command to deploy from your local machine. With render.com, it integrates with your Git provider and monitors pushes to the repository, and then initiates a deployment itself.

But with either mechanism, you want to have confidence when your change has been successfully deployed.

A number of factors might cause a delay before the change is publically visible. For example, caching by the hosting provider, or even your own caching layer, might mean that if they flag it is having been deployed, it might take a further few seconds or minutes before that is reflected across their CDN (something I’ve certainly noticed with render.com).

The examples below will offer some more concrete certainty based on Git commit hashes. Exposing and then testing your site against an expected Git hash being available is the key to this method.

Exposing the Git hash - static vs dynamic sites

A simple example for a dynamic site is to offer a version check endpoint at a known URL route. Here you allow your deployment pipeline to pass in a commit hash based on the commit that is being deployed, and the app will check this against it’s known deployed commit hash and return either:

a 404 status indicating that version is not yet deployed (not found)
a 204 success status, yet we don’t need to provide any content in the response

render.com offers an environment variable RENDER_GIT_COMMIT for the running application, and so an example looks like this for a Ruby based Sinatra app:

get '/version-check/:commit' do
    params[:commit] == ENV['RENDER_GIT_COMMIT'] ? 204 : 404
end

For a static site that is built with a tool such as Hugo, then you need to approach this a slightly different way. When the site is built, generate a static file named with the Git commit hash. We will then be able to check if the file exists, which will return either a 404, or 200 by the web server:

mkdir -p static/version-check
echo "ok" > "static/version-check/${GITHUB_SHA}"

With Hugo, files within the static directory are served as they are, unmanipulated, at the root of the project.

Both static and dynamic projects will use the URL format of:

/version-check/COMMIT_HASH

For example:

/version-check/df5222b2ac95769803a54cc44ef9698aff65be34

Testing for the presence of the correct commit

Once the PaaS solution has begun, or indicated the deployment is complete, you need to check for the presence of the latest commit hash. With GitHub Actions, you can use ${GITHUB_SHA} to determine this.

You can use the Node.js tool wait-on or curl to retry the URL until it receives a 2xx (successful) status. The curl option might be more appropriate if you don’t have or want to setup Node.js within your deployment pipeline.

For wait-on, you can use this in your pipeline run steps:

npx wait-on --timeout 600000 "https://yourapp.com/version-check/${GITHUB_SHA}"

This will wait 600,000 miliseconds, or 10 minutes, before exiting in failure.

And for curl, you can use:

curl --head --retry 600 --retry-all-errors --fail --retry-delay 1 "https://yourapp.com/version-check/${GITHUB_SHA}"

After these commands have run successfully, then you’re free to run any subsequent commands that you want to trigger.

GitHub Actions full examples

For my blog, I use something similar to the below for a Hugo build handled on Fly.io:

name: deploy

on:
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3
        with:
          # Hugo uses submodules for themes:
          submodules: 'recursive'
          # I use `enableGitInfo` for Hugo:
          fetch-depth: 0

      - name: create version check file
        run: |
          mkdir -p static/version-check \
            && echo "ok" > "static/version-check/${GITHUB_SHA}"

      - uses: superfly/flyctl-actions/setup-flyctl@master

      - name: deploy to fly
        run: flyctl deploy --remote-only
        env:
          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

      - uses: actions/setup-node@v3
        with:
          node-version: 18

      - name: wait for deployment to be active
        run: npx wait-on --timeout 600000 "https://davidwinter.dev/version-check/${GITHUB_SHA}"
        id: deploy_wait
        continue-on-error: true

      - name: send Telegram success message
        if: steps.deploy_wait.outcome == 'success'
        uses: appleboy/telegram-action@v0.1.1
        with:
          to: ${{ secrets.TELEGRAM_CHAT_ID }}
          token: ${{ secrets.TELEGRAM_BOT_TOKEN }}
          format: markdown
          message: |
            🎉 davidwinter.dev deployed!

            Git SHA: `${{ github.sha }}`

            https://davidwinter.dev

      - name: send Telegram fail message
        if: steps.deploy_wait.outcome != 'success'
        uses: appleboy/telegram-action@v0.1.1
        with:
          to: ${{ secrets.TELEGRAM_CHAT_ID }}
          token: ${{ secrets.TELEGRAM_BOT_TOKEN }}
          format: markdown
          message: |
            🚨 davidwinter.dev failed to deploy

            Git SHA: `${{ github.sha }}`

And for a dynamic site deployed to render.com, you can see a small example application written in Ruby with Sinatra here: https://github.com/davidwinter/render-post-deploy

Simple Electron automatic updates

2020-10-31T00:00:00+00:00

For my note-taking application, nimblenote, I was looking for the simplest way to handle automatic updates for the user. I was keen to be in a position whereby when I publish new versions of the app that its users were onboarded to that new version as quickly as possible. Ensuring people aren’t stuck on old versions of your app is essential, especially when it comes to fixing any bugs, and dealing with support issues.

I use the amazing electron-builder package, which handles the packaging and publishing of nimblenote. It offers a simple auto-update mechanism via electron-updater, however, it uses your operating-system based notifications which are often missed when they appear, or sometimes snoozed or disabled entirely.

Operating system notifications weren’t going to cut it for me. I wanted to be able to present a dialog box which would allow the user to easily relaunch and update the application with one button click. Although electron-updater does offer all you need to do this via an extendable API, it was going to result in around 100 lines of code to achieve what I wanted—and also—that wasn’t going to be easy to reuse, unless… I packaged up the functionality into a package!

Introducing electron-auto-update, which I’m very pleased has had almost 1,000 downloads in its first week!

This package works in tandem with electron-builder, which I highly recommend for packaging and publishing your application.

Install via either yarn add electron-auto-update or npm add electron-auto-update and then use in your application with:

const {autoUpdate} = require('electron-auto-update');

autoUpdate();

This will automatically check for updates every hour, and if available, present a simple dialog:

The user can choose to be reminded later, or opt-out of any subsequent reminders if they choose. The check frequency for updates can also be configured, along with also passing through a logger object.

Head on over to the project on GitHub for the full README, and if you find electron-auto-update useful in your projects, please consider sponsoring me on GitHub ♥.

Managing GPG with Keybase

2019-04-25T00:00:00+01:00

GPG is a tool used for encryption. Similar to SSH, you have a GPG public and private key. Your private key you keep to yourself, and it is used to encrypt and decrypt messages based on a public key that you share with others.

I’ve always found it quite confusing to manage and setup, and to keep keys in sync amongst computers, however, with some more recent need to use it, I hope that I now have something I can use day-to-day without issue.

Keybase is a tool that has been around for a number of years now. Originally it had GPG at it’s heart, but has since moved to it’s own encryption protocol which aims to make encryption easier to use. However, it still supports GPG under the hood too.

At the end of this blog post, I’ll show you how you can sign your Git commits using GPG to add an extra layer of security for other users to ensure your work is from you. And in a later blog post, I’ll discuss how to use pass in a software project for managing and sharing secrets between your team.

You’ll want to make sure you have GPG and Keybase installed before proceeding.

Setting up Keybase and GPG

I’m using a Mac to do this, and most of these instructions should still work.

You’ll want to visit https://keybase.io/download and download the Keybase application for the platform you’re using. Once installed run the application and sign up. This should also add the keybase command line program too.

Run:

keybase login

Once you’re all logged in, and your first device has been added to your account, you’ll want to create your first GPG key. Out of the box, Keybase protocol for encryption, but it does support GPG too. And seeing as this is what we want to use for Github commit signing, and pass down the line too, we’ll go this route.

keybase pgp gen --multi

Follow the prompts and enter the answers to the various questions that you’re asked. When it gives the option of storing your private GPG key on Keybase, I opted for yes, as it makes using it on other devices and computers easier.

Great, you now have a GPG key!

Publishing your key outside of Keybase

While Keybase is a great solution, and it stores your public key for other users to find too, not everyone uses Keybase. GPG can be used outside of Keybase using the gpg tools. To make your key available to those users, you’ll want to publish your public key to some of the more popular GPG key servers. This way, other users can pull down your key and use it to verify and decrypt things from yourself.

Each GPG key you have is represented by a fingerprint. This is a 40 character hexadecimal string. To retrieve this, you can type:

gpg -K

You’ll see some output like:

sec   rsa4096 2019-04-17 [SC] [expires: 2035-04-13]
      3AB5D447C087FB4DD80E3FA40194134D61D5C337

Copy your own long fingerprint and use that in the following commands:

gpg --send-keys 3AB5D447C087FB4DD80E3FA40194134D61D5C337
gpg --keyserver pgp.mit.edu --send-keys 3AB5D447C087FB4DD80E3FA40194134D61D5C337
gpg --keyserver keyserver.ubuntu.com --send-keys 3AB5D447C087FB4DD80E3FA40194134D61D5C337

Along with the standard GPG keyserver, this will also upload it to the MIT and Ubuntu servers that are commonly used too.

Other users would then be able to retrieve your key by using:

gpg --recv-keys 3AB5D447C087FB4DD80E3FA40194134D61D5C337

Using your key on your other machines

If you have multiple computers, you’ll want to setup the key on other machines too. As you’re storing both your public and private key on Keybase, you can easily do this.

After you’ve installed Keybase and GPG on your additional machines, you can run:

keybase pgp export -q 3AB5D447C087FB4DD80E3FA40194134D61D5C337 | gpg --import
keybase pgp export -q 3AB5D447C087FB4DD80E3FA40194134D61D5C337 --secret | gpg --allow-secret-key-import --import

Signing your Git commits

Signing your Git commits is basically adding your own signature to each commit to let others know they have definitely come from you. This is quite important as anyone can configure Git to use your email address, but not everyone has your GPG private key.

If you visit your Github settings, then on the left hand side select SSH and GPG Keys, then click on New GPG key. You’ll want to paste in the public part of your GPG key. You can retrieve this by typing:

keybase pgp export

Once you’ve clicked on Add GPG key Github will be able to verify commits from you. Now you need to tell Git on your computer to use this key to sign commits.

Open up your Git config located at ~/.gitconfig and under the [user] section, you’ll want to add:

signingkey = 3AB5D447C087FB4DD80E3FA40194134D61D5C337

So it should look like:

[user]
  name = David Winter
  email = i@djw.me
  signingkey = 3AB5D447C087FB4DD80E3FA40194134D61D5C337

This tells Git which key to use. To enable signing, add the following also:

[commit]
  gpgsign = true

All of your commits from now on will be signed by your GPG key.

Working remotely with SSH and VNC

2019-03-15T00:00:00+00:00

Visiting the house out in Spain is one of our favourite things to do in the off-peak seasons. When it’s grey, gloomy and wet back home in the UK, a weeks visit is all you need to top up on Vitamin D and 20-degree heat.

With a solid 4G connection hooked up the house, remote work is a possibility for me as I just need that connection for web development. My better half needs to do data modelling and has just bought a 16 core PC back home that he uses as a mini-supercomputer. His MacBook out in Spain wouldn’t get him far, so I set about setting him up so that he could utilise the processing power back home while sitting by the pool out in Spain.

Accessing home IP address

With time of the essence before our departure to the airport, I needed a quick and easy way to gain access to our home IP address as it’s not a static one and could change at any point. The PC at home would be on all of the time, so I created a small shell script that would run hourly and output the IP address into a Dropbox folder so that it would be retrievable from Spain.

sudo nano /etc/cron.hourly/home_ip

#!/bin/bash

dig +short -4 myip.opendns.com @resolver1.opendns.com > ~/Dropbox/home_ip.txt

sudo chmod +x /etc/cron.hourly/home_ip

This would now run daily and Dropbox would keep this in sync for me.

SSH

Simple enough on Ubuntu:

sudo apt install openssh-server

Copy over your SSH key from the computer that you’ll be connecting to your home PC with:

ssh-copy-id <username>@<local-ip-address-of-home-pc>

It will prompt you for your password, but once the command has completed, you will be able to SSH in without a password.

If the command tells you that you don’t have an SSH key, run:

ssh-keygen -t rsa

You can hit enter to accept all of the defaults, but you may want to assign a passphrase to your key for extra security.

Port forwarding

This can be different depending on your home router, but it’s essential to forward a public port on your router onto the SSH service running on your home PC. For us, we decided on a non-standard SSH port. For example, port 2200 would forward onto port 22 of our home PC.

In your router settings, also try and fix the DHCP assigned IP address of your home PC, or set it to static if there is an option just to eliminate the risk that the IP address may change on your local network, which would then break the port forwarding.

Testing SSH connection

It’s a good time to now test the SSH connection outside of your local home network. You can tether to your phone, or visit somewhere else with a different connection.

You can try the connection with:

ssh -p 2200 george@$(cat ~/Dropbox/home_ip.txt)

Remember to substitute the port for the one you selected in your router settings and your username. The $(cat ~/Dropbox/home_ip.txt) will be swapped out for the contents of the file that contains your home IP address.

You should then have a successful connection via ssh.

VNC for remote desktop

This is an optional step, but if you want to have a remote desktop experience too, you can enable VNC. In Ubuntu, under sharing preferences, you’ll be able to enable this. It’s important to set a password to allow for a connection that doesn’t require a person at the computer to manually approve the connection. This would not be helpful while out in Spain!

By default, the Ubuntu VNC server, Vino, has encryption enabled. That’s great! However, the built-in macOS VNC client does not support this, so we need to disable it on Ubuntu with:

gsettings set org.gnome.Vino require-encryption false

Now on your local network, you can test the connection. From the Finder menu, select Go > Connect to Server or push cmd + k to open the dialog, then you can enter:

vnc://<your-local-ip-address>

You’ll be prompted to enter your passphrase and then you can connect and should be able to see the screen share.

If you trust your local network users, not having encryption enabled is probably fine. However, I would not recommend now opening the VNC port and use port forwarding to gain access to this remotely as it would not be encrypted over the wider internet.

Instead, let’s use SSH tunnelling so that we gain encryption again while also being compatible with the macOS VNC client.

SSH tunnel for VNC

We already have our SSH port forwarding on our router, so remotely, we just want to set up an SSH tunnel for the local VNC port.

ssh -N -L 5200:localhost:5200 -p 2200 george@$(cat ~/Dropbox/home_ip.txt)

The important parts of this command are 5200:localhost:5200 which tells your computer to tunnel your 5200 port with the port 5200 on your localhost machine (in this case the home PC). Everything after this is just the standard SSH connection to the home PC.

So now, you can connect to your home PC from your Mac with:

vnc://localhost

And this will be tunnelling to your home PC via the SSH connection, with encryption out of the box.

Tmux - the essentials

2019-03-14T00:00:00+00:00

Tmux is a great tool for managing multiple terminal sessions and layouts. You can disconnect from a tmux session and then reconnect to it later and carry on where you left off. There is a vast amount you can configure with tmux, and many commands that can be used, and at first it can be intimidating to learn and some may fear a steep learning curve. But knowing only a handful of essential commands is enough to be productive day-to-day. I’ve been using tmux for the past two years or so and haven’t really ventured beyond these, and still, have found plenty of value out of it.

Sessions

If you’ve never run tmux before and have no open sessions, just type:

tmux

This will open a new window with a single pane filling the entire screen.

One of the features of tmux is the ability to detach from a running tmux session and then be able to reconnect to it at a later time with everything being as you’d left it.

To detach from the new session you’ve just created type ctrl + b and then d. You will now have been returned to your original command prompt.

You can see the tmux session that you were just connected to by running:

tmux ls

To reattach to it, type:

tmux a

Tmux prefix command

Let’s talk about the tmux prefix command; ctrl + b. All commands you run within a tmux session are initiated with this. You press this combination first, and then after, the command you want to run. From here on, we’ll reference the prefix with prefix rather than the full ctrl + b.

Windows and panes

When within tmux there are two basic concepts that you need to know about if you’re going to use it productively; windows and panes. You can think of windows as tabs within a tmux session. They are listed along the bottom of the tmux window. Each window can have multiple panes, which are different terminal prompts split across the window in various layouts.

Windows

Each window is prefixed with a number to represent it. Type ctrl + b and then c to create a new window. You’ll see it appear in the window list and it has been selected. You now have two windows, one represented by 0 and the other by 1.

You can switch between windows by typing prefix + 0 to jump to the first window. Just use the number index to choose which window you want to switch to.

When you start creating lots of windows, it becomes hard to remember what is within each. By default each window will be named with the default shell it is running. To rename, type prefix + , and then enter a new name, then hit enter. You’ll see the name reflected in the window list displayed along the bottom of the screen.

Panes

Within window 0, let’s create some panes. Type prefix + % to split the current window into two vertical panes. You’ll notice the right-hand pane currently has the terminal focus.

Type prefix + q and the number for each pane will briefly appear on the screen. This is how you will navigate between panes. The numbers will disappear after a second or so. Just push the combination again prefix + q and then type 0 immediately to switch focus back to the leftmost pane.

Now go back to the rightmost pane with prefix + q and then 1. We can split this pane in two horizontally by pressing prefix + ".

Press prefix + q again and you can now see we have three panes indexed with 0—2.

Sometimes it’s very helpful to zoom in on a single pane. A good example is when you want to copy and paste the contents of a pane that go over more than one line. To zoom press prefix + z. You’ll notice the pane expands to fill the entire window. Also a Z will appear in the window list along the bottom. To unzoom, just press prefix + z again.

Summary of commands

tmux create a new tmux session
tmux ls list any existing tmux sessions
tmux a reattach to the last open tmux session
ctrl + b the default tmux command prefix
prefix + d detach from current tmux session
prefix + c create a new window
prefix + 0-9 to switch to the numbered window
prefix + , rename the existing window
prefix + % split the current pane into two vertical panes, left and right
prefix + " split the current pane into two horizontal panes, top and bottom
prefix + q view numbered panes for current window
prefix + q, 0-9 switch to pane immediately after displaying pane numbers
prefix + z to zoom and unzoom

Quick and simple Github Gist oEmbed support

2014-10-13T00:00:00+01:00

WordPress doesn’t come with Github Gist oEmbed support out of the box, which is a shame because sometimes there is nothing quicker than creating a Gist to put in a new tech post by just pasting in the URL.

Well, it is quite simple to drop in with a WordPress hook that I’ve written:

<?php
wp_embed_register_handler(
    'github-gist',
    '/^https:\/\/gist\.github\.com\/([^\/]+)\/([^\/]+)/',
    function($matches, $attr, $url, $rawattr) {
        $embed = "&lt;script src="\"{$url}.js\""&gt;&lt;/script&gt;";

        return apply_filters(
            'embed_github_gist',
            $embed,
            $matches,
            $attr,
            $url,
            $rawattr
        );
});

Add that code to your themes functions.php file and then when you’re using the visual editor, if you paste in any Github Gist URL, it’ll automatically be converted.

Install and manage WordPress with Composer

2014-10-12T00:00:00+01:00

Well over two years have passed since my previous blog post on managing WordPress with git was written. A lot has changed since then. Composer has taken the PHP world by storm and the majority of developers are seeing the benefits and embracing it.

Using Composer, you can list a set of requirements for your project, and let it figure out where everything needs to be installed to, what dependencies are required, and how to obtain them.

If you’re not familiar with Composer, I would recommend that you go and read the Getting Started guide. This howto assumes you have knowledge of, and have perhaps used Composer before. This howto will cover installing WordPress with Composer. Following blog posts will cover installing themes and plugins.

Installing WordPress

First, you need to make sure you have Composer installed for your project, this is easily done by running the following command:

mkdir my-wordpress-project
cd my-wordpress-project
curl -sS https://getcomposer.org/installer | php

You will now have a composer.phar file in your project directory.

Create a new composer.json file with the following contents:

{
    "repositories": [
        {
            "type": "package",
            "package": {
                "name": "wordpress",
                "type": "webroot",
                "version": "4.0",
                "dist": {
                    "type": "zip",
                    "url": "https://github.com/WordPress/WordPress/archive/4.0.zip"
                },
                "require" : {
                    "fancyguy/webroot-installer": "1.0.0"
                }
            }
        }
    ],
    "require": {
        "wordpress": "4.*",
        "fancyguy/webroot-installer": "1.0.0"
    },
    "extra": {
        "webroot-dir": "public/wp",
        "webroot-package": "wordpress"
    }
}

A quick explanation of what the above is doing for us. WordPress doesn’t support Composer installs out of the box - it doesn’t provide a composer.json file. So we have to create a virtual Composer package for WordPress. We give it a name, version and location to a zip file archive where it can be downloaded from.

We don’t want the package to be installed to the default vendor directory that Composer would use. We need to specify a different directory. We do this with the aid of the webroot installer package. By giving our virtual package a type of webroot, the plugin will detect and relocate the install location of WordPress to public/wp.

We define the preferred install located in the extra section of the composer.json file.

We need to create the public directory:

mkdir public

Note: It’s good practice to have a separate directory in your projects for your document root. This is so that if you have any other files, that shouldn’t be served via the web server, you can place them outside of the public directory.

You can now run:

php composer.phar install

This will pull down WordPress for you into the correct location.

Now with a normal install you’d go ahead and proceed with the famous 5 minute install. But because we want to be able to use Composer for any future WordPress updates, we don’t want to change any of the contents of the public/wp directory. We want to be able to ignore this directory with our source code management tool.

In order to be able to use WordPress, we need to copy some files from the public/wp directory to the public directory.

cp -R public/wp/{wp-content,index.php,wp-config-sample.php} public/

mv public/wp-config-sample.php public/wp-config.php

In public/index.php, modify the following line:

require( dirname( __FILE__ ) . '/wp-blog-header.php' );

With:

require( dirname( __FILE__ ) . '/wp/wp-blog-header.php' );

We’re just updating the path as we want to reference the core WordPress files in the public/wp directory.

Go to https://api.wordpress.org/secret-key/1.1/salt/ and update the lines in your public/wp-config.php file:

define('AUTH_KEY', 'put your unique phrase here');
define('SECURE_AUTH_KEY', 'put your unique phrase here');
define('LOGGED_IN_KEY', 'put your unique phrase here');
define('NONCE_KEY', 'put your unique phrase here');
define('AUTH_SALT', 'put your unique phrase here');
define('SECURE_AUTH_SALT', 'put your unique phrase here');
define('LOGGED_IN_SALT', 'put your unique phrase here');
define('NONCE_SALT', 'put your unique phrase here');

That’ll secure your WordPress install. Configure your database settings as you normally would.

Now we need to set a few additional constants for the configuration.

define('WP_CONTENT_DIR', __DIR__ . '/wp-content');
define('WP_CONTENT_URL', 'http://' . $_SERVER['SERVER_NAME'] . '/wp-content');
define('WP_SITEURL', 'http://' . $_SERVER['SERVER_NAME'] . '/wp');
define('WP_HOME', 'http://' . $_SERVER['SERVER_NAME']);

Because the wp-content directory isn’t in the same place as the core WordPress files (remember, it is now one directory level above), we need to tell the config file where it actually is. The same with the core WordPress files.

Some other optional tidy up; remove the default twenty* themes:

rm -rf public/wp-content/themes/twenty*

Remove the default Hello Dolly plugin too:

rm public/wp-content/plugins/hello.php

You probably have an idea of what theme you’ll be using. Setting this constant will define the default theme, so that you don’t need to manually set this in your admin. This is optional.

define('WP_DEFAULT_THEME', 'mytheme');

Regarding housekeeping with your source code management tool. You’d want to ignore the composer.phar file and public/wp directory. Everything else can be committed and pushed.

Updating WordPress

Say you want to update to version 4.0.1 when it is released. In your composer.json file, update the section under repositories so it looks like:

{
    "type": "package",
    "package": {
        "name": "wordpress",
        "type": "webroot",
        "version": "4.0.1",
        "dist": {
            "type": "zip",
            "url": "https://github.com/WordPress/WordPress/archive/4.0.1.zip"
        },
        "require" : {
            "fancyguy/webroot-installer": "1.0.0"
        }
    }
}

We’ve updated the version number and the URL for the zip archive to reference the new versioned file.

You can then run:

php composer.phar update wordpress

Then commit your composer.json and composer.lock files.

That’s all there is to it. This is quite a stable workflow, and my next blog posts will cover handling the installation of plugins and themes.

Introduction to Ansible

2013-11-23T00:00:00+00:00

I’ve been using Ansible now for just under six months. I love it. I used to use Puppet religlously, but with a change of job, I needed something simple to setup, to work with a range of different servers, and be easy for others to learn.

Puppet is very powerful, and I was very happy with it. I still use it for some of my projects, because it just doesn’t make sense to switch them over to Ansible unless there is a specific reason that warrants rewriting the manifests.

You may think Puppet will win over people who manage multiple servers, and that maybe Ansible can’t handle this^*. You’d be absolutely wrong to make that assumption. Ansible is known to be used in setups managing in the range of 20,000 nodes. Ansible does’t require a complex master-slave setup, as Puppet does, that takes in itself hours to setup. And it’s coped perfectly with any setup I’ve encountered so far.

Provide Ansible with a list of machines to provision, and the specifics and it’ll just go ahead and do it.

Lets dive in.

Note: I’m using Vagrant for this demo. Though, all of the Ansible specific parts are not dependent on it.

Also, at time of writing, I’m using Vagrant 1.3.5 and Ansible 1.3.4.

Install Ansible with:

sudo easy_install pip
sudo pip install -U ansible

Get a base Ubuntu machine up and running:

mkdir ansible-demo
cd ansible-demo
vagrant init precise64 http://files.vagrantup.com/precise64.box

We’ll want to give the machine a dedicated IP address that we can have Ansible connect to, so in your Vagrantfile, ensure you have the following line:

config.vm.network :private_network, ip: "192.168.33.20"

config.vm.provision :ansible do |ansible|
  ansible.playbook = "private/ansible/site.yml"
  ansible.inventory_path = "private/ansible/development"
end

We’ve also told Vagrant that we want to use Ansible to provision our box.

Let’s create the directory structure for Ansible:

mkdir -p private/ansible

Now to tell Ansible about the machine we want it to provision. Add the following file private/ansible/development and add the following:

[development]
192.168.33.20

This adds our new Vagrant machine to the development group. We specify it’s IP address so Ansible knows how to connect to it.

Note: An inventory file isn’t actually required for Vagrant—it would create one automatically for us if we hadn’t—however, it’s a core part of Ansible, so worth using it with an explanation.

Now we’ll tell Ansible how to provision our Vagrant machine. Create the file private/ansible/site.yml and inside it add:

---
- hosts: all
  sudo: yes
  tasks:
    - name: Update Apt cache
      apt: update_cache=yes

    - name: Install Apache
      apt: pkg=apache2 state=present

Lets get the machine up and running:

vagrant up

After a while, you should see some output like:

[default] Running provisioner: ansible...

PLAY [all] ********************************************************************

GATHERING FACTS ***************************************************************
ok: [192.168.33.20]

TASK: [Update Apt cache] ******************************************************
ok: [192.168.33.20]

TASK: [Install Apache] ********************************************************
changed: [192.168.33.20]

PLAY RECAP ********************************************************************
192.168.33.20              : ok=3    changed=1    unreachable=0    failed=0

If you now visit http://192.168.33.20 in your browser, you should see the amazing “It works!” Apache default host file.

If you hadn’t noticed already, site.yml is written in YAML. To run through the site.yml:

- hosts: all

This allows you to target specific groups for Anible to apply rules to. We’ve used the development group in the private/ansible/development file. You could also have webservers, databases, etc. For our introduction, we’ll just tell Ansible to target all hosts.

As we’re installing packages on Ubuntu, we need root privileges, so we tell Ansible to use sudo.

Next up we define a bunch of tasks to run. Tasks use Ansible modules to apply things to the machine. Each task you give a descriptive name which is used in the output when Ansible runs. Be as descriptive as possible.

We use the apt module. First of all, we tell Apt to update it’s cache so that it can install new packages.

Then we get the apache2 package installed. We tell Ansible to ensure it’s present so that on subsequent runs, it doesn’t need to try and install it again.

So… the Apache ‘It works!’ page isn’t that much fun. Lets get PHP up and running.

Create a file called test.php in the directory private/ansible/files with the contents:

<?php phpinfo();

At the bottom of site.yml we’ll include some more tasks:

- name: Add test php file
  copy: src=files/test.php dest=/var/www/test.php

- name: Install PHP
  apt: pkg=libapache2-mod-php5 state=present

We have two new tasks here. The first will copy the local test.php file to the remote machine into the specified destination. We then install the PHP module for Apache.

If we were to provision this now with Ansible, the packages would be installed and the file would be copied, however, we need to restart Apache so that it loads the PHP module.

We do this in Ansible with handlers.

At the bottom of site.yml add the following:

handlers:
  - name: Restart Apache
    service: name=apache2 state=restarted

We then want to reference this handler when we install the PHP module. Update the PHP apt task to be:

- name: Install PHP
  apt: pkg=libapache2-mod-php5 state=present
  notify: Restart Apache

We’ve added the notify part to the task. This will tell Ansible to run the Restart Apache handler when PHP is installed, which in turn will restart the Apache server making PHP become available.

At this stage it’s safe to run vagrant provision.

Once this has run, you’ll be able to visit http://192.168.33.20/test.php and you should see a PHP info page.

That’s nice. But we’ll be a bit more exciting.

In your Vagrantfile lets mount out Vagrant project to the virtual machine so that we can use files in our project. Add this below the :private_network line:

config.vm.synced_folder ".", "/vagrant", :nfs => true

This mounts our Vagrant project directory, denoted by the period . and on the virtual machine, mounts it to /vagrant.

Create a new file called wordpress.conf and put it inside the private/ansible/files directory with the following contents:

<VirtualHost *:80>
    DocumentRoot /vagrant/public
    ServerName wordpress.dev
</VirtualHost>

Go to WordPress.org and download the latest version. Extract the files and move the wordpress directory into your project and rename it to public.

Now add the following to your site.yml file below where we installed PHP:

- name: WordPress virtualhost
  copy:
    src: files/wordpress.conf
    dest: /etc/apache2/sites-available/wordpress.conf

- name: Enable WordPress virtualhost
  file:
    src: /etc/apache2/sites-available/wordpress.conf
    dest: /etc/apache2/sites-enabled/wordpress
    state: link
  notify: Restart Apache

- name: Disable default Apache virtualhost
  file:
    dest: /etc/apache2/sites-enabled/000-default
    state: absent

- name: Install MySQL
  apt: pkg=mysql-server state=present

- name: Install python-mysqldb
  apt: pkg=python-mysqldb state=present

- name: Install PHP MySQL bindings
  apt: pkg=php5-mysql state=present
  notify: Restart Apache

- name: Setup WordPress database
  mysql_db: name=wordpress state=present

It should be pretty clear what this does by just reading the name titles for each task! However, I’ll explain two of them.

We disable the default Apache virtual host so that we don’t have to worry about setting up a hostname in our /etc/hosts file. This is something you’ll more than likely do when you have a few different projects on your machine, but we’ll skip that for this demo.

We also install the python-mysqldb package. This is installed so that Ansible can then create databases for us with the mysql_db task. It’s an Ansible dependency that we have to include so that we get to use Ansible more.

Because we’ve changed the Vagrantfile we can’t just do a vagrant provision - we need to actually reboot the machine so that Vagrant can mount the /vagrant directory share. Because we’re using NFS for this, it’ll need sudo privileges, so you may be prompted for your password.

vagrant reload

Once the machine is running again, provision with Ansible:

vagrant provision

You can now visit http://192.168.33.20 and you should be preented with a WordPress installer. That’s it. Very little effort. You can use the default MySQL credentials to setup Wordpress:

Host: localhost
User: root
Password: leave this blank
Database: wordpress - this is what we specified in our site.yml file

Note: We’re not worrying about securing MySQL. This is just a development virtual machine, so we don’t have to worry about it. That’s why we use the root account with no password.

That’s a very quick, brief introduction to Ansible. In my next post, I’ll talk more about reusable Ansible tasks - roles.

Some very helpful websites to have in your arsenal when working with Ansible:

Ubuntu Packages for finding package names to use to install. Be sure to select Precise as your Ubuntu version which is Ubuntu 12.04 LTS
Ansible Modules documentation has all of the references for modules you can use

Update: Michael DeHaan—creator of Ansible—thought I should clarify my original sentence. Quite rightly so, as it could have been misinterpreted that I was suggesting Ansible couldn’t handle complex/large/master-slave setups. It can do all of them. The point I was trying to make is that it’s something Puppet users may think it can’t, and that they could make decisions not to use Ansible because of a misunderstanding.

@davidwinter "and that maybe Ansible can’t handle this" ... might wish to clarify we have users in the 10k-20k node range, etc :)
— Michael DeHaan (@laserllama) November 24, 2013

Poor mans VPN with sshuttle

2013-08-10T00:00:00+01:00

It was far too hot in the office the other week and I needed some place a little cooler to work so I could keep focused. I sought refuge at the Google Campus in London which has glorious a/c and a–surprise surprise–super fast internet connection.

However, being in a basement full of geeks, using Google’s wifi, and not encrypting your connection, is quite a risky thing to do. I get paranoid about people sniffing my traffic.

I’m a subscriber to Private Internet Access, which is a VPN provider. I use that with OpenVPN using Tunnelblick for OS X. I’ve been using it fine for ages and have had no issues. Google doesn’t appear to like me encrypting my traffic. I’m able to connect to PIA, but after a while, my wifi connection drops. And then I’m not able to reconnect to Wifi unless I restart my Mac. I’ve never had this issue before or anywhere else.

Setting up an SSH SOCKS proxy has always been quite reliable and easy way to setup an encrypted connection. I’ve used it for bypassing my old University’s firewall so I could connect to IRC. Also, for getting around annoying image compression on my old 3G connection from o2. Though recently I’ve noticed not all traffic is guaranteed to go over the SOCKS proxy, so it’s not as secure as using a VPN.

After lunch, I moved onto The Book Club. The wifi was no way near as fast, but the food in there is amazing! They also had an annoying Firewall setup. I couldn’t connect to PIA via the VPN either. I setup the SOCKS proxy again but because not all of the traffic goes over the proxy, apps such as Spotify, Hipchat and some others just didn’t want to work. Not having some sort of encryption wasn’t an option.

With a bit of Googling, I come across sshuttle. It’s written in Python and does some fancy SSH’ing, and allows you to route all traffic over it, without the need of setting up an SSH SOCKS proxy.

brew install sshuttle

You just run the command, and it just works. It’s amazingly simple:

sshuttle --dns -vvr david@my.remote.server.com 0/0

Just replace david@my.remote.server.com with your remote server that you can SSH into. sshuttle will handle the rest. When you’re done with the connection, just CTRL + c.

Switch to a retro shell, Fish shell

2013-08-04T00:00:00+01:00

I’ve been using fish shell for the last year, and would never go back to using bash or zsh again. Ever.

All I want is for my shell to be speedy, and helpful. As standard. Without the need to install a mountain of plugins, or have a 200 line configuration file.

Fish is perfect for me. And I’m sure it’ll be a perfect match for you too. Some highlights:

Invalid commands; if you mistype mkdir and type in mkdi or something similar, the command will appear in red. Instant feedback.
Valid filepaths appear underlined.
Autosuggest; while typing a command, autosuggest will appear in grey to the right of your cursor.
Autosuggest knows about paths and options, and will also pull in your history too.

Your config is in a bit of an obscure location (the only negative I can think of) at ~/.config/fish/fish.config.

If you’re moving from bash or zsh, probably the most important thing you’ll want to set, is your $PATH variable. Here’s how in fish:

set PATH "/usr/local/bin" $PATH

This is just prefixing /usr/local/bin to the front of my $PATH so my homebrew works.

Set your default EDITOR?

set -g -x EDITOR vim

Want to display your current branch when in either a git or Mercurial repo?

function git_prompt
    if git root >/dev/null 2>&1
        set_color normal
        printf ' on '
        set_color magenta
        printf '%s' (git currentbranch ^/dev/null)
        set_color normal
    end
end

function hg_prompt
    if hg root >/dev/null 2>&1
        set_color normal
        printf ' on '
        set_color magenta
        printf '%s' (hg branch ^/dev/null)
        set_color normal
    end
end

function fish_prompt
    set_color $fish_color_cwd
    printf '%s' (prompt_pwd)
    set_color normal

    git_prompt
    hg_prompt

    echo ' >: '
end

This gives you a really clean prompt with just a retro Lost looking >: prompt.

There’s not really much else I want to write about. This was more of a ‘spread the word’, evangelical post. Just take the dive, and switch. You won’t regret it.

Go and read more in the fish tutorial.

Identity theft and no one seeming to care but me

2013-04-09T00:00:00+01:00

This post is a little out of the ordinary for my blog - well, OK, quite a bit. But I thought it was worth posting in the hope that other people who have had this happen to them can get in touch. If you’re expecting another tech related post, don’t bother to continue reading!

I woke up last Thursday to a letter in the post from Vodafone. Strange I thought, seeing as though I had cancelled my contract with them ages ago and was now on a new network.

The letter was a confirmation of setting up a direct debit. It had my name and address (obviously). The bank account used for the direct debit was in my name, however the sort code and account number weren’t mine. So that was a relief right away knowing that my account hadn’t been drained.

Someone had ordered a phone using my name and address, and either setup a bank account in my name, used some other poor persons or just faked the details.

I did a search for the bank via the sort code which resulted in a Derby branch of Lloyds TSB. I definitely don’t have a Lloyds account and don’t think I’ve ever been to Derby.

Visited the Vodafone website and saw that they had online live chat. Not had much success with these previously–usually someone in an office reading off a script–never usually helpful. I let the support person know the situation. The letter had on it the account number of this new account so I gave them that. They were able to block the account and the phone within a few minutes.

They were also able to tell me that the phone had been bought in a shopping centre local to where I live. Interesting. Not just some random scammer in a far away town, but actually someone in close proximity to me. I straight away thought someone had been through my rubbish and got my details from that, but then reflected thinking how paranoid I am about these sorts of things, and therefore shred 99.9% of all post I receive. There was a possibility someone had found something of mine, but a low one I think. So then my mind wanders to think perhaps it is someone I know. Lots of people know my address, I have a few ‘enemies’, and considering how close the shopping centre is to me that the phone was bought at, I think that is a higher probability.

I asked the live chat person how someone can buy a phone in a retail store. They said they’d need some sort of ID with them. I asked whether that may be photo ID and they said yes. The stores make a copy of these when the purchase is made. So if someone had used photo ID it’d have been with their face and my name on it.

The support person suggested I go to the store and talk to the Manager there to see the ID, beacuse I said it may be someone I know. Also the store would most likely have CCTV and I had the time and date the purchase was made. Surely a simple case of reviewing the CCTV footage and asking me to see if I recognised the person.

The support person (Shakib K) was so very helpful. I was surprised—but grateful—that I was able to contact someone so quickly and painlessly. The Vodafone account had been blocked and the support person had submitted an internal fraud report to Vodafone that would be investigated, and I passed on my details so they could contact me.

Before heading to the Vodafone store, I tweeted Lloyds TSB asking them who I should contact about the fraud. The first reply I had just said I needed to go into a branch. Unhelpful and not convienient considering I’m trying to help them and their customer. I replied back asking if there was a telephone number I could contact. I got another reply with a number to call, which I did.

I explained the situation, gave them the sort code and account number. Interestingly, the name they had for the account wasn’t mine, but someone elses. So someone hadn’t created a fake account of mine, but was instead using someone elses. But how the hell was Vodafone and Lloyds TSB able to create the direct debit if the names of the account holder didn’t match? Surely something is wrong with that process. Sounds so easy for the criminal to do.

When I suggested to them that they should contact the account holder to make them aware that someone may be using their bank account to steal money, they said they coudln’t do anything. Nothing at all. Zip.

Ridiculous.

I suggested that they should see if a Vodafone direct debit existed on the account and should cancel it. I had the date the confirmation had been made.

Nope, couldn’t do that either. They said I had to go into a branch and report it.

What the hell? You have someone calling them up, at what I assume was their fraud department, or something similar, explaining that a dodgy direct debit had been setup on an account of theirs, and they don’t do anything? No action would be taken.

They expect the person reporting it to go out of their way, waste their time reporting it? Yes, I wouldn’t have minded, but after the unhelpful responses on twitter and on the phone, I decided against it. I had pleaded with them to at least flag the account. But they wouldn’t.

I understand they can’t delete the direct debit just because I called up. I understand they can’t just phone the customer and say someone has phoned up saying money has been taken out of their account. But they could at least add a note to the account so that if the account holder does call up, they have some information or clue as to why money has been stolen from them. They could at least write a letter saying that they suspect someone has setup a fraudulent direct debit. The account holder could at least then check to see if that is the case, and call them to get assistance, or ignore it if nothing of the sort has happened.

Wouldn’t you like your bank to tell you as soon as possible if their is something suspicious? Or someone has reported something? I sure as hell would.

Lloyds TSB didn’t seem to be concerned at all, and weren’t bothered about their customer. They obviously prefer to be reactive rather than proactive in supporting their customers. I certainly won’t bank with them, or recommend them to anyone in the future.

So after that useless help from Lloyds TSB, I made my way to the Vodafone store. I took the letter with me, took my passport and a utility bill so that I had some genuine proof of ID with me.

I got to the store and had to wait around 10 minutes to see the manager. I explained the situation, and said that the online chat Vodafone rep had suggested I come to the store. I offered to show them the letter but the manager didn’t seem that interested. Because I had told them that no money had been stolen from me personally, that they couldn’t show me the ID used to make the purchase. Even though this was someone who had impersonated me and I had the letter to prove it. I was one of the victims.

I had had such excellent support from the online chat staff, but the feeling I got from the manager was that he wasn’t bothered at all. He obviously seems to prefer spending more time and care for people impersonating and stealing from others, than someone trying to report the situation and help apprehend and prosecute the criminal. Crazy.

The Police station was very close to the shopping centre, so I decided to go their to report the incident. I was aware that there was an online form to do the same thing, but because the shopping centre was so close to me, I thought the local police might be interested as it could possibly be someone I know. I ended up waiting 30 minutes and the desk clerk telling me to report it online. Dead end there. More time wasted.

Disillusioned, I head back home and look at the online fraud reporting form. At the very start, it says that the submission may not result in the crime being investigated. Oh that really encourages me to waste more time filling out the form to try and help catch a criminal. At least spring it on me at the end of me completing the form! Regardless, I completed the form. I got a crime number but I don’t expect anything to become of it.

The annoying thing is that there is probably CCTV evidence of the culprit in the Vodafone store, and a copy of the fake photo ID that was used. Isn’t that solid evidence and enough to get a prosecution? I’d have thought so. I thought it was worth 5 hours of my time chasing around reporting things to different people, but other parties obviously don’t.

OK, maybe I wouldn’t recognise the person on the CCTV or photo ID, but isn’t it worth a shot? And if not, why can’t Vodafone or the Police post the picture online? It may only be small fish, but if you get a positive ID from someone, arrest the individual and maybe it’ll put them on the right path or just simply prevent them from doing it again in the future.

By not doing anything, you encourage them to steal more, and encourage me less to report anything in the future - to the Police or companies involved. It’s completely backwards and not a good outlook for the future of peoples confidence in reporting crime.

Setting up a Puppet Master and Agents on EC2

2012-12-08T00:00:00+00:00

Now you’ve mastered how to use the EC2 command line tools (Ninja, right?), you’ll want to manage all of those newly created instances in the easiest way possible. Puppet is the answer. If you’re not familiar with Puppet, back in March I wrote an introduction to it which would be good to read before continuing. If you are familiar, read on.

Previously I’ve only used Puppet ‘agentless’, or simply, standalone. This is great for developing small sites that don’t require multiple instances, however with larger projects, you’ll need a master so that you can leave that to deal with the ‘agents’ that connect to it.

Master and Agents

The example we’ll use for this post is for a web hosting project. You want to create a new instance for each customers, with a web server and database. Those packages will be installed and configured centrally from one master server so that all the instances are identical in that sense.

When first connecting an agent to a master, they have to exchange SSL certificates so that they know who is talking to who. The default way to do this is to manually check and sign the requests of agents as they attempt to connect to the master. This is time consuming an inefficient.

Instead, we’ll tell the Puppet master to automatically sign all incoming agent requests. That may sound like a security risk because anyones Puppet agent instance could try and connect. But we’ll use EC2 security groups to protect our server. For each new web server instance we create, we’ll assign it a security group called puppets. Then on our Puppet master, we’ll only allow other instances to connect to Puppet who are in the puppets group. This means we can safely allow the master to auto-sign requests, knowing that only our web server instances are allowed to do so.

The Puppet master

We’ll need to configure the master so that it’s ready to auto-sign puppet agents.

IP address and DNS

We need a dedicated Elastic IP for our Puppet master:

ec2-allocate-address

The output of that command will print out an IP address. It’s not currently associated to an instance, but it’s there ready to associate.

Each Elastic IP address is also given a public facing DNS name. It’s in the format of:

ec2-<IP-ADDRESS>.<REGION>.compute.amazonaws.com

So if the IP we were allocated was 123.456.789.012 and we are using the eu-west-1 EC2 region, the DNS name would look like:

ec2-123-456-789-012.eu-west-1.compute.amazonaws.com

Note: The periods in the IP address have been replaced with dashes.

The clever thing with these DNS names is that when you use them to communicate between EC2 instances, the DNS name resolves to the private IP address of the instance associated to it. This means you’re keeping the traffic internal to EC2 and it’s not going out onto the internet.

Why’s this important? Because when you stop an instance, the IP is unassociated from it, and next time you start the instance, the private IP address is not guaranteed to be the same. Using the DNS name, you know it’ll always use the correct private IP address by the DNS name being resolved.

Note: To make your systems more flexible in future, and not so tied to a DNS name with an IP address in it, it’s worth setting up a DNS CNAME record on your domain that forwards onto the Elastic IP DNS name, such as puppetmaster.mydomain.com. That way, in your systems, you’re using your domain name, and you only then need to change the DNS CNAME record in future if you change the Elastic IP.

Security groups and key pair

Create the security group that our Puppet agents will use.

ec2-create-group puppets -d "Puppet agents"

Don’t worry about assigning ports to this group yet, at this stage, we just need to ensure it exists.

Now create the security group for the Puppet master:

ec2-create-group puppetmaster -d "Puppet master"

For the puppetmaster group we want to allow TCP port 8140 to only the puppets security group:

ec2-authorize -P tcp -p 8140 -o puppets puppetmaster

We’ll throw in SSH too as it’s always handy to login just to see how things are:

ec2-authorize -P tcp -p 22 puppetmaster

Generate a new key pair for the master, and save it to a file, and set permissions:

ec2-create-keypair puppetmaster | sed 1d > puppetmaster.key
chmod 600 puppetmaster.key

Master instance and setup

Almost ready to create the instance, but when we initially do, it’ll be a base install of Ubuntu with packages that we need not installed, such as Puppet master.

A great feature of EC2 is that of user data scripts. These are scripts that are run when the instance is first created. We need to create a script that installs and configures Puppet master.

Create the following file and call it puppetmaster.sh:

#!/bin/bash
set -e -x
EC2_HOSTNAME=ec2-123-456-789-012.eu-west-1.compute.amazonaws.com
export DEBIAN_FRONTEND=noninteractive
hostname $EC2_HOSTNAME
echo $EC2_HOSTNAME > /etc/hostname
aptitude -y install puppetmaster
echo "*" > /etc/puppet/autosign.conf
service puppetmaster restart

With the above, the only line you need to change is the value of the EC2_HOSTNAME variable, which should be either the DNS name of the Elastic IP or the CNAME record you created at your domain.

The script will set the hostname of the instance, and then install Puppet master. The order of this is important, because when Puppet master is installed, it uses the hostname of the instance to create an SSL certificate for the master to use when auto signing agent certificates. If you have an incorrect hostname, the Puppet agents will complain and fail when trying to communicate with the master.

We then also tell Puppet master to autosign all requests, and then restart the service for the changes to take affect.

Now create the EC2 instance with the following:

ec2-run-instances --user-data-file puppetmaster.sh -t t1.micro -g puppetmaster -k puppetmaster ami-3b65664f

The instance will not be created, and started, and the user data script will run. In the output of this command, you’ll get details of the instance that has been created, along with an instance ID that starts with a prefix of i-. Use that ID to associate the Elastic IP to the instance:

ec2-associate-address -i i-INSTANCEID 123.456.789.012

Where 123.456.789.012 is the Elastic IP address you was allocated earlier.

Your master server is now setup and ready.

Setup your manifests

Although puppet master is ready to go, we don’t currently have any manifests setup. So for this example, we’ll just do a basic web server setup.

SSH into the master:

ssh -i puppetmaster.key ubuntu@ec2-123-456-789-012.eu-west-1.compute.amazonaws.com

Puppet master, by default, stores it’s manifests and modules at /etc/puppet. We’ll create a simple manifest that installs nginx and adds an index.html file to the default virtual host.

sudo nano /etc/puppet/manifests/site.pp

In the manifest file, add:

package { 'nginx':
    ensure => present,
}

service { 'nginx':
    ensure  => running,
    require => Package['nginx'],
}

file { '/usr/share/nginx/www/index.html':
    content => '<h1>Hello from Puppet master!</h1>',
    require => Package['nginx'],
}

Now when we boot up new Puppet agent instances, Puppet master will install nginx on them, and update the default index.html file with our changes.

The Puppet agents

You’ve already created a security group for these instances, so we’ll just authorise SSH and HTTP traffic for it.

ec2-authorize -P tcp -p 22 puppets
ec2-authorize -P tcp -p 80 puppets

Create a keypair for the puppet agents to use:

ec2-create-keypair puppets | sed 1d > puppets.key
chmod 600 puppets.key

We need to create another user data script for the agents in order for them to install Puppet when they are first created. Create a file called puppets.sh and put in it:

#!/bin/bash
set -e -x
EC2_HOSTNAME=ec2-123-456-789-012.eu-west-1.compute.amazonaws.com
export DEBIAN_FRONTEND=noninteractive
aptitude -y install puppet
echo "
[agent]
server=$EC2_HOSTNAME
" >> /etc/puppet/puppet.conf
sed -i /etc/default/puppet -e 's/START=no/START=yes/'
service puppet restart

Again, you just need to update the EC2_HOSTNAME variable to point to the DNS name (or custom CNAME) pointing to the Elastic IP for the master server. The rest of the script installs Puppet, and points the Puppet agent to the master server. It also sets the Puppet agent to start when the instance is started or rebooted.

Now you can create an instance:

ec2-run-instances --user-data-file puppets.sh -t t1.micro -g puppets -k puppets ami-3b65664f

Give the instance enough time to be created and booted. The following is then happening:

Puppet is being installed
It’s being configured to use our puppet master
The puppet agent requests to the puppet master, which accepts
Puppet master sends over the manifests (and modules if you specify any) and starts executing them

In our case, that means it’ll install nginx and our test index.html file.

So that we can check it has worked, run ec2-describe-instances and in the output, find the instance that has the security group puppets. That’ll be the one we just created. We want the public DNS name. Copy that, and paste it into a browser and you should see after a few minutes once it’s had time to setup:

Hello from Puppet master!

Our Puppet agent has connected and hooked up to the master successfully. Now you can continue to create other agent instances knowing that they’ll connect up the same. Those instances can then all be managed by the Puppet master.

Success.

Install EC2 command line tools with OS X

2012-12-07T00:00:00+00:00

I’ve been using EC2 a lot recently, probably creating and destroying around 20 instances trying out a project, today alone. I needed to be able to do this a lot faster than the web console lets me, so I downloaded the EC2 command line tools to give me that bit extra.

Installing

It’s very easy to setup:

Grab the zip file from the AWS developer site
Unzip to your home directory
Rename the extracted directory to .ec2
Go to the Security Credentials page on the AWS site
Select the X.509 Certificates tab
Click on Create a new Certificate
Download the two files in the popup to your ~/.ec2 directory

Configuring to work with your CLI

Now we need to configure our shell–Fish in my case–to use the EC2 tools. Copy the following into your ~/.config/fish/config.fish file:

set PATH $EC2_HOME/bin $PATH
set -g -x JAVA_HOME /System/Library/Frameworks/JavaVM.framework/Home/
set -g -x EC2_HOME ~/.ec2
set -g -x EC2_URL https://eu-west-1.ec2.amazonaws.com
set -g -x EC2_PRIVATE_KEY (ls {$EC2_HOME}/pk-*.pem)
set -g -x EC2_CERT (ls {$EC2_HOME}/cert-*.pem)

If you’re using bash or zsh, it’s a similar setup - just create the variables, and update your PATH. Here’s bash (add this to your ~/.bash_profile file):

export PATH=$PATH:$EC2_HOME/bin
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home/
export EC2_HOME=~/.ec2
export EC2_URL=https://eu-west-1.ec2.amazonaws.com
export EC2_PRIVATE_KEY=`ls $EC2_HOME/pk-*.pem`
export EC2_CERT=`ls $EC2_HOME/cert-*.pem`

Note: I’ve set my default region to eu-west-1. You may want to set this to something different. Change the EC2_URL variable accordingly.

Open a new tab or terminal window and the changes should now be present. To check, you can run:

ec2-describe-instances

You should get a list of the instances currently associated to your account in your default region.

Taking the tools for a spin

A good way to test the tools is to use my previous post on getting started with EC2, but rather than using the web AWS console, we’ll use nothing but the command line tools.

Create the key pair:

ec2-create-keypair blog | sed 1d > blog.key

The output of this file includes a fingerprint of the key at the top, which we don’t want, so we use sed to remove that line so we can easily save it to a file. We then just set restrictive permissions:

chmod 600 blog.key

Create the security group:

ec2-create-group blog -d Blog
ec2-authorize blog -P tcp -p 22
ec2-authorize blog -P tcp -p 80

Create and run the instance:

ec2-run-instances -t t1.micro -g blog -k blog ami-e9eded9d

From the output of this command, get the instance ID, which is prefixed with i-.

Assign a dedicated IP:

ec2-allocate-address

The above command will output an IP address. Copy that to your clipboard, and use it in the following command (paste it in place of 123.456.789.012):

ec2-associate-address -i i-INSTANCE_ID 123.456.789.012

You can then ssh into the instance with:

ssh -i blog.key ubuntu@123.456.789.012

All done.

Getting started with Amazon EC2

2012-12-06T00:00:00+00:00

A few friends have asked me to talk through how to setup an Amazon EC2 server.

At first glance, it’s very intimidating, confusing, and not quite clear where to start. But once you know about all the pieces that bring together the AWS services, it’s not difficult at all to get a server up and running.

The great thing about EC2 is that the servers are virtualised. That means you can create and throw them away when not needed very easily. This makes it great for testing out things, not having to worry that you’ll have to pay out for a full server for a month. Boot up an EC2 server, run it for a few minutes, or non-stop. It’s up to you. You have complete freedom and flexibility.

Ready to tuck in?

Some definitions

We’ll start off with defining some of the core pieces that you’ll need to be able to create, and then run an EC2 server:

EC2 & Instances

EC2 stands for Elastic Cloud Compute. This service allows you to setup virtual machines (known as instances) to run an operating system of your choice (mine is Ubuntu).

EBS

Elastic Block Store, these are the virtual hard drives your instance will use.

AMI

Amazon Machine Image, these are best described as base installs of an operating system, with a default configuration set and ready for the EC2 service.

Security Groups

This is where you can setup what is essentially a firewall for your instance. You say what ports are open on it. This means in the majority of cases you don’t need to setup a software firewall on your install (such as iptables or ufw). It saves a lot of headaches.

Key Pairs

These are the SSH keys that you’ll use to connect to your instance. When you create the machine, the public key of the selected pair is already added to the authorized_keys file, and you then download and use a private key to connect.

Elastic IPs

Allows you to assign a dedicated IP to the instance. Not required, but helpful for when ready to launch, and want to point a domain name.

Getting started

Before you can do anything, you need to have an AWS account. Register if you don’t already have one, then head to the AWS Console.

I’ve already mentioned that Ubuntu is my OS of choice, so we need an AMI to base our instance on. I always use the official Cannonical (who manage Ubuntu) ones, and they have a helpful webpage for you to find the correct AMI-ID. You need this ID in order to tell EC2 which AMI to use.

I want to use the latest version of Ubuntu (12.10 at time of writing), with a 64 bit processor, based in the EU region (which is in Ireland) with an EBS hard drive.

To narrow down the list of available AMI’s, I type in ‘12.10 amd64 eu ebs’ into the search box. We’re then left with one AMI - ami-e9eded9d.

Creating the instance

You’ll notice that the ID is a link which you can click on to start creating your EC2 instance. It directs you onto the AWS console.

The first screen gives you a summary of the AMI and the defaults for the hard drive setup - which is 8GB. This can be increased at a later time if needed. Click ‘Continue’.

The important part on this step is selecting the size of the instance. It should be defaulted to ‘T1 Micro’. Amazon offer a variety of sizes that increase the power and RAM for your instance. The bigger the instance, the more you pay. Again, these can be changed at a later point. Ensure ‘T1 Micro’ is selected and then continue.

We won’t be choosing any advanced options, so skip the next step. We’ll also leave the default hard drive configuration, so skip the following step too.

The next screen allows you to name the instance. You’ll see a key with the name prefilled ‘Name’, and you can set the value to something descriptive. If you’re running a blog on the instance perhaps call it ‘blog’. Click ‘Continue’.

When the instance is running, you’ll need to be able to connect to it via SSH so you can configure it. This is where SSH key pairs come into play. You’ll want to create a new key pair. You just need to give it a name, and then click on the ‘Create & Download your Key Pair’. Depending on what you call the Key Pair, for example, ‘blog’, a file called blog.pem will download. Keep this somewhere safe and secure.

Note: It’d be a good idea to add a passphrase to this key file so that if someone did get the key, they couldn’t use it to connect to your server without the passphrase. For this demonstration, we’ll leave it as is, without. There’s a good writeup on Github on why and how to use and create them.

Next we need to configure the Security Group for the instance. Create a new one and give it a name and description. Then you can either choose from the dropdown for common port configurations, or set something custom. We’ll enable SSH (so we can connect to it remotely) and HTTP (because we’ll probably be running a web server) for this instance. Select both of these, and add them to the Security Group. Click ‘Continue’.

You’re now on the final step before creating and starting the instance. You can check all of the configuration options we’ve set from here, and then you’re ready to launch. Click ‘Launch’.

AWS will tell you that your instance is now launching. Click on ‘Close’.

In the left hand menu of the console, select ‘Instances’. You should see a ‘running’ status for the instance you just created it. Select it from the list.

In the details section of the page near the bottom, you’ll see your instance has been selected, and it will give you a hostname you can use to connect to your instance with. It’ll look something like ec2-12-345-67-890.eu-west-1.compute.amazonaws.com.

Before we connect, you need to give the private key you downloaded earlier strict, private permissions for your user only.

chmod 600 path/to/blog.pem

Now you can connect to your instance:

ssh -i path/to/blog.pem ubuntu@ec2-12-345-67-890.eu-west-1.compute.amazonaws.com

You’ll notice the user I’m connecting with is called ubuntu. This is the default user that the AMI we’re using has setup. You should be promted to add the host to your known hosts file, so type in yes and hit enter.

And then, you’re in! Connected to your brand new instance, with a fresh install of Ubuntu 12.10 on it.

Give your instance a dedicated IP

You can now logout from SSH, and then we’ll assign a dedicated IP to the instance, which you’d then be able to use to point a DNS A record to the instance.

In your AWS console, select Elastic IPs. Select ‘Allocate New Address’ and EC2 should be selected in the dropdown. Click ‘Yes, Allocate’. With the new IP selected, click on ‘Associate Address’, choose the instance you’ve just created and then click on ‘Yes, Associate’.

Copy that new IP address to your clipboard, and then you can connect to your instance again with that:

ssh -i path/to/blog.pem ubuntu@123.456.789.012

In this example, we’re using 123.456.789.012 as that newly created IP. You should get the known hosts prompt again, type yes, and then you’ll be connected again.

That’s it. You’re ready to go.

Aditional notes

You pay for what you use on AWS. And the pricing is a bit complicated because of the level of detail at which Amazon charge you for. Processing time, bandwidth used, disk space consumed, IP addresses, etc etc.

A handy calculator is available to work out how much you’re likely to pay: http://calculator.s3.amazonaws.com/calc5.html.

While you’re developing with AWS, it’s probably a good idea to turn off your instance while you’re not using it. You can do this from the console by right clicking on the instance and clicking on ‘Stop’. To boot it up again, do the same but select ‘Start’ instead. If you want to delete the instance because you won’t be using it again, select ‘Terminate’. That’ll delete the data on the instance, so terminate carefully.

Install and manage WordPress with Git

2012-04-09T00:00:00+01:00

I’ve been a massive fan of the ‘Installing/Updating Wordpress with Subversion’ instructions on the WordPress codex for a good few years. If you have ssh access to a server and are managing a couple of WordPress sites, keeping them up-to-date with the latest version is easy with just one svn command.

However, with a bigger WordPress project in the pipeline, I need to be able to manage my themes and plugins in a git repository, while at the same time keep a reference to the core WordPress files, and allow it to be as easy to update when a new version is released.

I’m not going to continue with the SVN instructions because that’d add a lot of mess to my git repository - including SVN history. I want to keep track of my changes only - none of the WordPress related ones.

The good news is, this is all possible. It takes a few minutes longer to setup, but once done, updating will be as easy. You’ll also have a really tidy git repo.

Setup

Note: You can view the finished project on Github; davidwinter/wordpress-with-git

Create a new project directory:

mkdir mysite && cd mysite
git init
touch README.md
git add README.md
git commit -m "Initial commit."

You now have a blank project to start with.

We’ll be using the Github WordPress mirror that is synced with the official SVN repository every 30 minutes. It includes all tags and branches.

We want to add this as a subrepository to our main project.

git submodule add git://github.com/WordPress/WordPress.git wordpress

WordPress is quite a big project if you haven’t already noticed, so it might take a while to make a clone. Once done, commit the new subrepository:

git commit -m "Add WordPress subrepository."

We want to ensure our project is using the current stable version. As of writing, this is 3.3.1.

cd wordpress
git checkout 3.3.1
cd ..

This has now updated the WordPress repository to 3.3.1 - commit the changes to your main project.

git commit -am "Checkout WordPress 3.3.1"

The subrepository is now isolated, and we don’t want to make any changes to it, besides bumping version numbers. This means we can’t modify wp-config.php, themes or plugins. We’ll have to move things about a bit.

First up, we’ll create our config file. WordPress allows for this to be stored one directory up from the core WordPress files.

cp wordpress/wp-config-sample.php wp-config.php
git add wp-config.php
git commit -m "Adding default wp-config.php file"

Now that this is outside of the WordPress repository, we’ll be able to configure WordPress and commit the changes to our main project repository.

Now, in order to manage themes and plugins we need to make a copy of the wp-content directory:

cp -R wordpress/wp-content .

This will copy the directory and its contents into our main project repo.

Now, at this point you may like to tidy up that directory a little. This isn’t required, but will save you commiting themes and plugins you’ll probably never use.

rm wp-content/plugins/hello.php
rm -rf wp-content/themes/twentyten

If you plan on using the default WordPress theme, Twenty Eleven, don’t run the following command:

rm -rf wp-content/themes/twentyeleven

Now commit this directory:

git add wp-content
git commit -m "Adding default wp-content directory"

We now have a trimed down wp-content directory ready for only our themes and plugins.

To finish off the project structure, we need to copy the WordPress index.php file:

cp wordpress/index.php .
git add index.php
git commit -m "Adding index.php"

Now we need to configure these files and let WordPress know where to talk to everything.

Configuring

We’ll start with index.php. This is the file all requests go via, and we just need to update a require statement. Find the following line:

require('./wp-blog-header.php');

And update it to:

require('./wordpress/wp-blog-header.php');

Commit:

git commit -am "Pointing index.php to the correct location"

Now into wp-config.php, open it up. Because we have the core WordPress files in a different directory to that of the index.php file, we need to let WordPress know about it. Add the following two constants to the file. I prefer just below the @package WordPress statement so my changes are near the top of the file:

 * @package WordPress
 */

define('WP_SITEURL', 'http://' . $_SERVER['SERVER_NAME'] . '/wordpress');
define('WP_HOME',    'http://' . $_SERVER['SERVER_NAME']);

This lets WordPress know that the core files are in the wordpress directory, but that the site is to be served at the root of the project directory.

Because we moved the wp-content directory out of the core WordPress directory, we also have to let it know about this change. Add the following after the previous define statements:

define('WP_CONTENT_DIR', $_SERVER['DOCUMENT_ROOT'] . '/wp-content');
define('WP_CONTENT_URL', 'http://' . $_SERVER['SERVER_NAME'] . '/wp-content');

And finally, if you’re not using the default twentyeleven theme, and want to specify the one WordPress should use instead:

define('WP_DEFAULT_THEME', 'mytheme');

Simply replace mytheme with the name of the theme you have. Be sure that you’ve placed it inside the wp-content/themes directory.

Now simply complete the database details as normal further down in the file, and then visit the project on your webhost and you’ll be up and running.

git commit -am "Update settings in wp-config.php"

Updating

Change into the WordPress subrepository:

cd wordpress
git fetch --tags
git checkout 3.3.2

Replace 3.3.2 with the correct version number.

Now commit the changes subrepository version to your main project:

cd ..
git commit -m "Update WordPress to version 3.3.2"

Helpful references

Introduction to Puppet

2012-03-04T00:00:00+00:00

Have you ever had that déjà vu feeling when setting up a new server? That you’ve done it all before, and that it’s really tedious to have to do it all over again? You’d rather be able to just hit a button to deploy your app and start showing it off to the world.

It’s exactly what happened to me, and at first I thought I’d just write a bash script with all the various steps that I had started noting down in a plain text file. Luckily, at the point I was considering this, I saw the release of a new project called Vagrant. It’s a tool that allows you to run a virtual machine locally, with a helpful shared directory to your files on your local (host) machine. It allows you to setup a self contained development environment without messing around with your system.

Reading through the documentation, I saw it mention Puppet and Chef which both appeared to be some sort of witchcraft that allowed you to automatically configure and setup these virtualised servers. Bingo. Easy server configuration management.

You don’t have to use Vagrant to use Puppet. It’s just a really easy way to test it out locally. For your production environments you’ll still use the same puppet manifests, but you’ll manage the running of those manifests a little different. I’ll be covering the Vagrant approach for this blog post.

What do I want to demonstrate? Just a quick demo of how to get Puppet to install Nginx and php-fpm on a server, serving a simple phpinfo() file. It should be enough to make you thirtsy for more.

Some quick steps for setup

You’ll need to have Virtual Box installed with the command line tools option. Then install the Vagrant gem on your computer with a simple

gem install vagrant

Now create a new directory on your machine where you’re going to work on this project.

mkdir testing-vagrant
cd testing-vagrant/

All Vagrant projects use a ‘box’ as a starting point for your project. It’s an virtual disk image of an OS installation that has all the dependencies to interact with Vagrant, and to run puppet manifests. Now, the one on the Vagrant website is Ubuntu 10.04 Lucid Lynx, and without some tinkering, it’s not as easy to get the latest versions of packages. So instead, I created my own Vagrant box for Ubuntu 11.10 Oneiric Ocelot using the open source tool VeeWee. It’s hosted on my public Dropbox folder, and to add this box to your vagrant setup, run the following:

vagrant box add oneiric32 http://dl.dropbox.com/u/11342885/oneiric32.box

Once this has been downloaded, you need to initialise your Vagrant project with this box:

vagrant init oneiric32

This will create a file named Vagrantfile in your project directory. This file has a bunch of different configuration options commented out. It’s a really helpful file to read through.

One configuration option you’ll need to set is the port forwarding of the webserver. Ensure this line is present:

config.vm.forward_port 80, 8080

This will forward on the port 8080 of your local machine to port 80 on the vagrant virtualbox. If you don’t have any other webservers running on your local machine, feel free to point this to port 80 directly.

Also, ensure these lines are present too:

config.vm.provision :puppet, :module_path => "private/puppet/modules" do |puppet|
  puppet.manifests_path = "private/puppet/manifests"
  puppet.manifest_file  = "base.pp"
end

This is the Puppet configuration. It tells Vagrant where the modules and manifests directories are, and the base manifest file.

And finally, create these directories in your project for storing the different Puppet configuration files we need:

mkdir -p private/puppet/{manifests,modules/nginx/files}

Playing with Puppet

To understand how to use Puppet, you need to think of a Puppet manifest as a file that defines the different resources your server requires. Resources in Puppet speak are things like users, groups, packages, commands, services and files–to name a few. We define these resources in a manifests file which you’ll place in private/puppet/manifests/base.pp.

Now the first thing we want to do with the base box is ensure that everything is up-to-date. The first resource we’ll define is an executable command to run. Basically an apt-get update. In your base.pp file add the following:

exec { 'apt-get update':
    command => '/usr/bin/apt-get update',
}

All resources follow this same structure:

resource_type { 'title':
    options => value,
}

The title is quite an important value, as it is often used as a default option value too. For example, with the user resource, the title you specify will be the name of the user to create. This can be overriden if you need to define a descriptive title that isn’t the name of the user you want to create. For the user resource, you’d override this with the name option. For example, the following would both create the same user:

user { 'david':
    ensure => present,
}

user { 'my descriptive title for my username':
    name   => 'david',
    ensure => present,
}

Note: Due to an issue with VeeWee not creating a required user group on the Oneiric box, you need to also tell Puppet to create the puppet group. This can be done with the following. Not doing this will result in errors:

group { 'puppet':
    ensure => present,
}

Back to our example. We now have a command resource that will run apt-get update. Once this has been run, we want to install the packages nginx and php-fpm. We want to ensure though that apt-get update has run beforehand. Now, with Puppet, it’s important to note that the manifest files are not run through sequentially as you might expect. Each time they are run, they can happen in a different order. You need to explicity set dependencies and requirements.

To install the nginx package, we’ll add the following:

package { 'nginx':
    ensure => present,
    require => Exec['apt-get update'],
}

A few things to note here. The package to install is being derived from the resource title. It defaults to this, or you can specify the option with the option key name. We tell puppet to ensure that it’s present on the system. We also say that this resource requires that the executable command apt-get update has already been applied.

When you reference other resources in a manifest file, the resource type has the first letter capitalised. Then you specify the title of the resource you’re referencing. This is where resource titles are used.

We’ll also do the same for the php-fpm package:

package { 'php5-fpm':
    ensure => present,
    require => Exec['apt-get update'],
}

Each of these packages come with a service that runs in the background, that we want to ensure is running all the time. We can make Puppet check this is the case with the following:

service { 'nginx':
    ensure => running,
    require => Package['nginx'],
}

service { 'php5-fpm':
    ensure => running,
    require => Package['php5-fpm'],
}

Notice the require dependencies? Whenever Puppet runs it will check that these are running, and if they’re not, for whatever reason, it will start them.

Ready to see some magic? Save the file and run vagrant up. Vagrant will boot up the virtual machine, and run the Puppet manifest on it. Once it’s finished visit http://localhost:8080 and you should see:

Welcome to nginx!

You’ve not had to ssh into the machine once or run anything on the command line of that VM.

Now in order to get PHP setup and running with nginx, we need to modify some of the nginx config files. We need to do the following:

Disable the default nginx virtual host.
Copy over a new virtual host for our vagrant setup that forwards on PHP requests to php-fpm.
Symbolically link the virtual host config file to the sites-enabled directory.

In the private/puppet/modules/nginx/files directory we created, add the following in a file called vagrant:

server {
    listen 80;
    server_name _;
    root /vagrant;
    index index.php;

    location / {
        try_files $uri /index.php;
    }

    location ~ \.php$ {
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_index index.php;
        include fastcgi_params;
    }
}

This is a standard nginx configuration file for a virtual host. It will use /vagrant on the virtual machine as the document root. This is a shared directory that Vagrant sets up to link to your project directory on your local machine.

Saving that file, we can now open up the base.pp file again and add a few more resources. The first is to copy over this virtual host to the correct location:

file { 'vagrant-nginx':
    path => '/etc/nginx/sites-available/vagrant',
    ensure => file,
    require => Package['nginx'],
    source => 'puppet:///modules/nginx/vagrant',
}

Now lets disable the default virtual host the nginx package provides:

file { 'default-nginx-disable':
    path => '/etc/nginx/sites-enabled/default',
    ensure => absent,
    require => Package['nginx'],
}

And finally, enable our new vagrant virtual host:

file { 'vagrant-nginx-enable':
    path => '/etc/nginx/sites-enabled/vagrant',
    target => '/etc/nginx/sites-available/vagrant',
    ensure => link,
    notify => Service['nginx'],
    require => [
        File['vagrant-nginx'],
        File['default-nginx-disable'],
    ],
}

Save the file and run vagrant reload. This tells vagrant to run the puppet provisioner again and ensure that the virtual machine matches the resources specified in the manifests file. We’ve just added three new ones, so it will action these.

Ready to test it has all worked as expected? Create an index.php file in your project directory, the same location where your Vagrantfile is. In there just put a simple:

<?php phpinfo();

Save the file. Visit http://localhost:8080 again and you should see the standard phpinfo page being served over php-fpm via nginx.

That’s it for this introduction. Hopefully it’s enough to get you excited. For me, it means being able to keep a manifest file up-to-date, and if I ever change my VPS to a different provider and want to get it up and running within a few minutes rather than hours, I can apply it with Puppet. No more keeping a log of the many steps I would have to manually run over ssh as the alternative.

If you want to know more, here are some great resources:

Testing Javascript websites with Behat

2012-01-14T00:00:00+00:00

This is my third blog post on how to use Behat for your web application testing. If you haven’t already, be sure to read Getting started with Behat and Using Behat with Mink as this post continues on where they left off.

In my previous post I left you with being able to create your own custom, reusable definitions in Behat. We’ve touched on the real basics of testing with Behat on a very simple example website. However, in 2012, you’ll find it very hard to find a website that doesn’t have some kind of Javascript included providing a dynamic and better user experience for visitors. It’s crucial that we’re able to test and guarantee that the animations and ajax requests that the users interact with work as we expect.

In this blog post, we’ll be using the twitter bootstrap example site. It has a variety of different JS related widgets and interactions which are perfect for explaining some of the basic Javascript testing concepts when it comes to Behat. Here is the behat.yml configuration file being used:

default:
  context:
    parameters:
      base_url: http://twitter.github.com/bootstrap/

Now take a look at the following example test:

Feature: Modal dialog

    Scenario: Open modal dialog
        Given I am on "/javascript.html"
        And I should see "Launch Modal"
        When I press "Launch Modal"
        Then I should see "One fine body…"

If you visit the modals section on the example site, there is a red button under the demo heading. Clicking that opens a modal box with the text ‘One fine body…’. That’s what our above test is for. Run that test and you’ll probably see:

Feature: Modal dialog

  Scenario: Open modal dialog            # features/modal.feature:3
    Given I am on "/javascript.html"     # FeatureContext::visit()
    And I should see "Launch Modal"      # FeatureContext::assertPageContainsText()
    When I press "Launch Modal"          # FeatureContext::pressButton()
      The selected node does not have a form ancestor.
    Then I should see "One fine body…" # FeatureContext::assertPageContainsText()

1 scenario (1 failed)
4 steps (2 passed, 1 skipped, 1 failed)

Great. Test failure. But we know it should work because we just did the exact same steps our test has in our browser. Why did it fail?

Behat by default uses a headless driver called Goutte for all tests. It’s a very fast driver, but it doesn’t support Javascript. As the official documentation for Mink so perfectly states:

Those browser emulators send a real HTTP requests against an application and parse the response content. They are very simple to run and configure, because this type of emulators can be written in any available programming language and can be run through console on servers without GUI. Headless emulators have both, advantages and disadvantages. Advantages are simplicity, speed and ability to run it without the need in real browser. But this type of browsers have one big disadvantage - they have no JS/AJAX support. So, you can’t test your rich GUI web applications with headless browsers.

We need to use a different type of driver for our Javascript-dependant tests. These type of drivers are called ‘browser controllers’. They launch the browser you designate and then it does exactly what your tests say. This means your tests have the full Javascript support of your browser behind them.

The easiest of these drivers to get setup and running is Selenium2Driver. We’re going to use this with Google Chrome to get our tests passing. Thanks to @ColonelRosa for pointing out Selenium2 support in Mink 1.3.

First of all you’ll need the chromedriver on your machine, and have it available in your path. You’re able to download the driver from the Google code project page. Or if you’re on a Mac and have homebrew installed, simply run brew install chromedriver.

The last thing you’ll need is the WebDriver. Go to http://seleniumhq.org/download/ and then you want to choose the ‘Selenium Server’. As of writing, it’s version 2.16.1. This will download a Java .jar file. Once you have this, simply run the following command in the terminal. It’s good idea to have this run in it’s own window or tab, because it must be running while the tests are active:

java -jar /path/to/selenium-server-standalone-2.16.1.jar

You now have all the required tools.

Now we just need to update our behat config, and our tests slightly. Update your behat.yml to contain:

default:
  context:
    parameters:
      base_url: http://twitter.github.com/bootstrap/
      browser: chrome
      javascript_session: webdriver

This is telling behat that for our javascript tests, we want it to use the WebDriver driver (Selenium2). Also, that it is to use Google Chrome as the browser the driver controls.

Now, we need to simply mark our test as being Javascript dependant, which is as easy as adding a @javascript tag above the Scenario:

Feature: Modal dialog

    @javascript
    Scenario: Open modal dialog
        Given I am on "/javascript.html"
        And I should see "Launch Modal"
        When I press "Launch Modal"
        Then I should see "One fine body…"

Now run behat.

Damn. The tests are still failing. This is because when the button is pressed, Javascript animates the opening of the modal window, and hasn’t completed by the time the next step runs searching for the text. The animation hasn’t completed, and therefore the step fails.

Well, we can get the test passing, and also add an additional definition that will make testing modals easier in the future. Lets update the scenario to the following:

    @javascript
    Scenario: Open modal dialog
        Given I am on "/javascript.html"
            And I should see "Launch Modal"
        When I press "Launch Modal"
        Then I should see the modal "Modal Heading"
            And I should see "One fine body…"

We’ve added a new definition I should see the modal "Modal Heading". This will allow us to write other tests that depend on modal dialogs opening with various headings. Run behat to get the skeleton definition method and add it to your FeatureContext.php file:

/**
 * @Then /^I should see the modal "([^"]*)"$/
 */
public function iShouldSeeTheModal($title)
{
    $this->getSession()->wait(20000, '(0 === jQuery.active && 0 === jQuery(\':animated\').length)');
    $this->assertElementContainsText('#modal-from-dom .modal-header h3', $title);
    assertTrue($this->getSession()->getPage()->find('css', '#modal-from-dom')->isVisible());
}

I’ll run through this method now line-by-line:

We use the wait method to allow us to wait for the animation to complete. The first parameter sets 20 seconds to wait or until the following Javascript evaluates to true. We test to ensure that all Jquery ajax calls have completed, and that no elements are being animated.
We want to check that the heading portion of the modal dialog contains the heading text that we pass through in the step. In our test, this is “Modal Heading”. So we use a CSS selector to do this.
Finally, we just want to be sure that the modal dialog is actually visible on the page.

That’s it. One final change we can make is to move our Jquery wait check code into it’s own method so that we can use this in any other custom definitions we may create in future:

protected function jqueryWait($duration = 1000)
{
        $this->getSession()->wait($duration, '(0 === jQuery.active && 0 === jQuery(\':animated\').length)');
}

/**
 * @Then /^I should see the modal "([^"]*)"$/
 */
public function iShouldSeeTheModal($title)
{
    $this->jqueryWait(20000);
    $this->assertElementContainsText('#modal-from-dom .modal-header h3', $title);
    assertTrue($this->getSession()->getPage()->find('css', '#modal-from-dom')->isVisible());
}

Now run behat, and you should get a perfect test score:

1 scenario (1 passed)
5 steps (5 passed)

Example code used in this post can be found on Github.

Using Behat with Mink

2012-01-13T00:00:00+00:00

This is my second blog post on how to use Behat for your web application testing. If you haven’t already, be sure to read Getting started with Behat as this post continues on where that left off.

In my last post, I left you with hopefully enough knowledge to get started with some basic testing that comes for free, out of the box, by just installing Behat and Mink. You were able to do that because Mink comes with around 34 bundled definitions.

We created a very basic test to ensure that users were able to log into our example website:

Feature: User sessions
    In order to access their account
    As a user
    I need to be able to log into the website

    Scenario: Login
        Given I am on "/"
            And I should see "Login"
        When I fill in "email" with "myemail@test.com"
            And I fill in "password" with "mysecurepassword"
            And I press "Login"
        Then I should be on "/dashboard"
            And I should see "Welcome back"

    Scenario: Logout
        Given I am on "/dashboard"
        When I follow "Logout"
        Then I should be on "/"
            And I should see "Login"

So that we can work on this further, I’ve created a basic PHP script that we can use to play with. It’s written with the Silex micro-framework and doesn’t require any modification in order for us to write the tests in this blog post.

If you do want to play along, clone the example from github, setup a virtual host. Then just be sure to update the behat.yml file so that the base_url points to your new virtual host.

If you run bin/behat now, it’ll run through the test and fail:

2 scenarios (1 passed, 1 failed)
11 steps (8 passed, 2 skipped, 1 failed)

In our test, we’ve missed something! For the second scenario, we’ve not logged in the user! So they’re unable to access the /dashboard.

If we focus on just the Logout scenario from here on, in order to log in the user, we could do something like this:

Scenario: Logout
    Given I am on "/"
        And I should see "Login"
    When I fill in "email" with "myemail@test.com"
        And I fill in "password" with "mysecurepassword"
        And I press "Login"
        And I should be on "/dashboard"
    When I follow "Logout"
    Then I should be on "/"
        And I should see "Login"

This is basically duplicating the steps from the above login scenario. Running this, the tests now pass. Excellent. Are we finished? No. The test above isn’t very tidy because we just copy and pasted code. It’d be so much nicer if we could write something like:

Scenario: Logout
    Given I am logged in as "myemail@test.com" with password "mysecurepassword"
        And I am on "/dashboard"
    When I follow "Logout"
    Then I should be on "/"
        And I should see "Login"

That makes our feature file a lot more readable, cleaner, and also provides us with a new custom definition that we can then use in other future tests. How do we make this work?

When you run bin/behat and you’ve added a custom definition, it’s very kind by providing you with an example template for creating the PHP code to have this new definition work:

/**
 * @Given /^I am logged in as "([^"]*)" with password "([^"]*)"$/
 */
public function iAmLoggedInAsWithPassword($argument1, $argument2)
{
    throw new PendingException();
}

What do we do with this? Paste it directly into features/bootstrap/FeatureContext.php.

Now we just need to use Mink to program the definition. Looking at the source for the base mink context, we can use the following methods to complete the definition:

/**
 * @Given /^I am logged in as "([^"]*)" with password "([^"]*)"$/
 */
public function iAmLoggedInAsWithPassword($email, $password)
{
    $this->visit('/');
    $this->fillField('email', $email);
    $this->fillField('password', $password);
    $this->pressButton('Login');
}

Saving, and then running bin/behat again, the tests will now pass. And you’ve just created your first custom definition.

There is one final change we could make to this. As our definition is using all existing definitions, we could future proof it by changing it to the following:

/**
 * @Given /^I am logged in as "([^"]*)" with password "([^"]*)"$/
 */
public function iAmLoggedInAsWithPassword($email, $password)
{
    return array(
        new Step\Given('I am on "/"'),
        new Step\When('I fill in "email" with "'.$email.'"'),
        new Step\When('I fill in "password" with "'.$password.'"'),
        new Step\When('I press "Login"'),
    );
}

In order to use the above, be sure to update the use statement at the top of the file so it contains the last line of the following:

use Behat\Behat\Context\ClosuredContextInterface,
    Behat\Behat\Context\TranslatedContextInterface,
    Behat\Behat\Context\BehatContext,
    Behat\Behat\Exception\PendingException,
    Behat\Behat\Context\Step;

That’s all there is to it when creating custom definitions. The Mink source is a great way to look at how the out-of-the-box definitions work.

Setup rbenv on Mac OS X

2011-12-10T00:00:00+00:00

rbenv is a quick way to get different ruby versions installed on your OS, without interfering with any default installs it may provide.

Run the following commands:

brew update
brew install rbenv
brew install ruby-build

In your bash profile file, ensure that the following is included:

eval "$(rbenv init -)"

Now, install the latest version of ruby:

rbenv install 1.9.3-p0

Run the following to set rbenv to use this new version of ruby.

rbenv global 1.9.3-p0

Open a new shell and type ruby --version and if you see ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-darwin11.2.0] then all is good and you’re all setup!

One thing to note is that if you install a new gem that installs a binary, you’ll need to run rbenv rehash afterwards in order for the binary to become available to you on the command line.

Getting started with Behat

2011-11-06T00:00:00+00:00

Where PHPUnit is for testing the individual pieces of your code (unit testing), Behat is a great tool for testing that an application behaves as expected when following a series of steps (functional testing). And for testing web applications, Behat ties in perfectly with Mink to give you a bunch of web browser related tests out-of-the-box.

Install

We’ll be using composer to setup Behat and Mink. Create a composer.json file in your project directory with:

{
    "require": {
        "behat/behat": ">=2.2.4",
        "behat/mink": ">=1.3.2"
    },

    "repositories": {
        "behat/mink-deps": { "composer": { "url": "behat.org" } }
    },

    "config": {
        "bin-dir": "bin/"
    }
}

If you don’t have the composer.phar file already on your machine, simply run:

wget http://getcomposer.org/composer.phar

Then run:

php composer.phar install

Behat and Mink will then install into a vendor and bin directory.

In your project directory, run:

bin/behat --init

This will create a features directory for you that contains the required files to get started.

In order to setup Behat to use Mink, go into features/bootstrap/FeatureContext.php and make sure that the following is defined:

require_once __DIR__.'/../../vendor/.composer/autoload.php';

And:

class FeatureContext extends Behat\Mink\Behat\Context\MinkContext

Here you’re including the Mink library, and then ensuring that FeatureContext extends from MinkContext. You don’t really need to worry about what this file does for this example.

Now let Behat know where it can test your web app from. In the root of your project directory, create a file called behat.yml and include the following:

default:
  context:
    parameters:
      base_url: http://localhost/mywebapp/

This is a YAML file, and you just need to update the base_url value to whatever URL behat can access your app.

You’re ready to start.

Describe your application with features

Run the following command:

bin/behat -dl

You should see a long list of available definitions that Behat currently has available to it. You can use these to write some powerful feature files that test your application.

First, lets set the scene:

Say on your homepage you have a login form and you want to make sure when someone logs in, they get directed to a dashboard page with a welcome message. You also want to check that when they logout, they are taken to the homepage with a confirmation message saying they’ve been logged out.

Each test file has the .feature extension that Behat uses to search for tests to run when calling the behat command.

These feature files are written in the Gherkin syntax. First of all you specify a particular feature you are testing. This is with the keyword Feature: and a name. After which, you add a brief description of what the benefit, the role, and the actual feature is. You can have anything here, but it’s good practice to follow this format to make things readable.

For each feature, you have a bunch of different scenarios that you want to test against to ensure it matches the expected behaviour. Scenarios consist of definitions that can be prefixed with either Given, When, Then and And. These keywords don’t mean anything, and are there purely to aid readability. Given is where you’d set a bunch of pre-conditions for your test, When for the steps taken to test, and Then should describe what expected result you should see. And can be used to include additional definitions per stage in a scenario.

Feature: User sessions
    In order to access their account
    As a user
    I need to be able to log into the website

    Scenario: Login
        Given I am on "/"
        And I should see "Login"
        When I fill in "email" with "myemail@test.com"
        And I fill in "password" with "mysecurepassword"
        And I press "Login"
        Then I should be on "/dashboard"
        And I should see "Welcome back"

    Scenario: Logout
        Given I am on "/dashboard"
        When I click on "Logout"
        Then I should be on "/"
        And I should see "Logged out"

Because we’ve written the steps in the available Mink definitions, we don’t need to write any PHP code! You can now run bin/behat in the Terminal and you’ll see the result of the tests and whether your application conforms to them–all lit up in green!

These example tests may seem rather trivial, but having them means you have a safety net for your application, so that if you or someone changes something, however small or large to your code, and it breaks the expected application behaviour, your Behat tests will let you know sooner, when it’s easy to fix, rather than later when it’ll become a nightmare to track down.

As an aside, it should be noted that Behat isn’t just great for us developers. If you’re working on a project for someone else, you can write up these feature files based on a clients spec, and get them to sign off of them for extra security and peace of mind for all parties.

With the available definitions that Mink provides for Behat, you could quite easily write a ton of tests with just the definitions Mink provides. In a future blog post I’ll go on further to show how to write custom defintitions. Hopefully this is enough information to get you started and excited about using Behat in your projects.

Setting up Github Pages

2011-10-29T00:00:00+01:00

I’ve not had time recently to keep my copy of WordPress up-to-date (yes, even though it’s setup with SVN). I wanted something a little easier to manage. Having read good things about Github Pages, I decided to take the plunge. I’ve now migrated this entire site over. It’s simple to setup, and also has Jekyll support enabled by default.

Jekyll allows you to generate a static website from plain text post files, and templates. When you push your new posts to Github, Jekyll is run on the files and the static site is built and hosted for you.

Setup

To get started, install jekyll:

sudo gem install jekyll

If you want to use Markdown formatting in your posts, also install:

sudo gem install rdiscount

Now you’ll need to create a git repository to manage your blog. There is a great Jekyll skeleton setup that I used here, so we’ll go ahead and clone that.

git clone https://github.com/danielmcgraw/Jekyll-Base.git yourgithubusername.github.com

Be sure you replace yourgithubusername with your Github username. This isn’t required on your local repo, but is for the repository you create on Github. It’s in this format so that Github can detect it’s a Github Pages site, and so it just makes sense to name it the same locally too.

Now, we don’t want the git history for this skeleton setup, so remove the .git directory:

cd yourgithubusername.github.com/
rm -rf .git

To test things are working at this stage, open another terminal window in the same directory and run jekyll --server. It keeps a process open, along with a webserver running on port 4000, monitoring file changes, and rebuilding the site automatically on the fly. So you’ll want to keep this terminal window open and running. You should now be able to see the generated site by visiting http://localhost:4000

When jekyll builds the site, it puts the generated files into a _site directory. You don’t want this directory in Git, so in the .gitignore file add _site to the bottom so that it’ll be ignored.

Now on Github, create a new repository naming it in the same format yourgithubusername.github.com. Then on your local machine:

git init
git add .
git commit -m "Initial commit. Jekyll base setup."

Then follow Github’s instructions on adding the remote reference, which is along the lines of (substitute username):

git remote add origin git@github.com:username/username.github.com.git
git push -u origin master

This pushes your site up to Github. After this initial push you can do a plain old git push when you’ve made changes that are ready to be published.

They say it can take up to 10 minutes for new sites to be generated when they are first pushed, however, it’ll likely be under a minute. They’ll email you either way when it’s done.

You can then visit http://yourgithubusername.github.com to see your new static site all built, and being served. Snappy isn’t it?

WordPress migration

There were two important things that I wanted to ensure kept the same during the migration to Github pages; the URLs for the posts, and the comments posted.

My permalink structure on WordPress was:

/articles/%year%/%monthnum%/%day%/%postname%/

So to set this up on Jekyll, all I had to do was edit _config.yml and set:

permalink: /articles/:year/:month/:day/:title/

For comments, I had already been using Disqus on my WordPress install, so they already had that content. All I had to do was setup the Javascript to pull in those comments. In the _layouts/post.html file, all I did was add:

<div id="disqus_thread"></div>
<script type="text/javascript">
    /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */
    var disqus_shortname = 'davidwinter'; // required: replace example with your forum shortname
    var disqus_url = 'http://davidwinter.me{% raw %}{{ page.url }}{% endraw %}';

    /* * * DON'T EDIT BELOW THIS LINE * * */
    (function() {
        var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
        dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
        (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
    })();
</script>
<noscript>Please enable JavaScript to view the <a href="http://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript>
<a href="http://disqus.com" class="dsq-brlink">blog comments powered by <span class="logo-disqus">Disqus</span></a>

The important stuff here is the disqus_shortname and disqus_url variables. These are required in order for Disqus to know which site comments to display, and the disqus_url to know which page comments to render. Because my URLs are staying the same, this was easy by just using the domain and the {% raw %}{{ page.url }}{% endraw %} variable.

While you’re setting this up locally, you may want to include:

var disqus_developer = 1;

As the help documents explain:

Tells the Disqus service that you are testing the system on an inaccessible website, e.g. secured staging server or a local environment. If disqus_developer is off or undefined, Disqus’ default behavior will be to attempt to read the location of your page and validate the URL. If unsuccessful, Disqus will not load. Use this variable to get around this restriction while you are testing on an inaccessible website.

Be sure that this is only set while testing the integration of Disqus. When you know it’s working locally, remove this variable, so that it doesn’t get pushed up to Github.

And also, ensure that your yourgithubusername.github.com address is added to your trusted domains list in the Disqus admin.

One last thing that is quite important to bring over to my new Github Pages setup is all of my old posts… duh. Depending on the size of your existing blog, you may just want to do this by hand. However, with 50+ posts, I wanted something automated.

In WordPress, you can do an export of your posts under the Tools section of the left hand menu. Just select to export the posts, not all content, as you don’t need it.

I wrote a quick PHP tool to convert this exported data into Jekyll friendly post files. Once these have been generated, you can copy them into your _posts directory, and Jekyll will generate the pages (if it’s still running).

Push up to Github, and you’re all set. Just tinker around with the layouts and styling and you’ll then have a super fast, easy to manage, Jekyll powered Github Pages site.

Simple local web development with Apache and Dnsmasq

2011-06-18T00:00:00+01:00

When you sit down and start work on a new project, what hoops do you have to jump through before you get your app running in your browser? There are more than likely always going to be two; creating a spoof domain that you create in /etc/hosts so that you can access your app, and then a virtual host in Apache to make your files accessible. And it sucks having to do this each time. Don’t you wish you could just get on with the project as soon as you think of it? Read on.

This was all inspired by Pow, a zero-configuration rack server created by 37signals, that allows Rails developers to do exactly that. By following convention over configuration, this can all be easily achieved. If you’re happy with accessing your apps via a spoof domain ending in .dev, then follow these steps.

Requirements

I’m assuming that you’ve got Homebrew installed, and that you’re using the built-in version of Apache on Mac OS X 10.6 Snow Leopard. However, these instructions should work fine on Linux too.

Dnsmasq

Dnsmasq is a simple to install and setup mini DNS server. We’ll use it to setup a wildcard record of anything .dev to point to your local machine.

brew install dnsmasq

Follow the instructions after install about copying the configuration file to the expected place in /usr/local/etc/dnsmasq, but don’t copy the launchd settings yet. First of all edit /usr/local/etc/dnsmasq and add the following at the very bottom:

address=/dev/127.0.0.1
listen-address=127.0.0.1

Save. Now copy the launchd settings across. Dnsmasq should now be running. You now need to tell your system to use it as a DNS server.

Open up System Preferences and go to your Network settings and the DNS servers. Add 127.0.0.1 to the top of the list, and then add whatever your normal internet facing DNS servers are beneath. I use OpenDNS. My DNS server list then looks like:

127.0.0.1
208.67.222.222
208.67.220.220

Apply changes. Now open the Terminal and run a ping on anything ending with a .dev:

ping test.dev

It should start pinging your machine, 127.0.0.1 - if it does, all is good.

Now we can move onto configuring Apache for the last time.

Apache

Edit the following file, substituting yourusername with whatever your system username is:

sudo nano /etc/apache2/users/yourusername.conf

And then make it look like the following, again replacing yourusername:

NameVirtualHost *:80

<Directory "/Users/yourusername/Sites/">
    Options Indexes MultiViews FollowSymLinks Includes
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

<VirtualHost *:80>
    UseCanonicalName off
    VirtualDocumentRoot /Users/yourusername/Sites/%0/public
</VirtualHost>

Save. And then run sudo apachectl graceful. This will restart Apache. In the above configuration, we’re using Apache’s mod_vhost_alias module to dynamically configure virtual hosts and document roots on the fly.

Testing it all works

mkdir -p ~/Sites/test.dev/public
echo "test.dev" > ~/Sites/test.dev/public/index.html

Now visit http://test.dev/ in your web browser. Hopefully you should see test.dev - if so congratulations; you’re going to save so much time in the future now.

Install APF on Ubuntu 11.04

2011-06-05T00:00:00+01:00

I just setup a new Linode to host my new twitter app, lists dj and wanted to lock it down to basically Apache and SSH.

iptables gives me a headache, and I don’t want to spend a day learning how to use it. APF uses iptables, but the configuration is so much easier.

sudo aptitude install apf-firewall
sudo nano /etc/apf-firewall/conf.apf

If you’re using Linode, you’ll need to ensure the following line and configuration value is set:

SET_MONOKERN="1"

This is because iptables is installed into the kernel rather than as a package. Not setting this will prevent APF from running.

Then, to open the desired ports, just update this line with the port numbers, separated with a space:

IG_TCP_CPORTS="22 80 443"

And ensure this line is set:

DEVEL_MODE="0"

This means that the firewall isn’t in development mode, otherwise your rules will be flushed every 5 minutes. Handy if you’re experimenting with new rules and don’t want to be locked out of your server. Worse case, you’d only have to wait 5 minutes before being able to get back in. But because I’m only opening ports, I don’t need this. Just make sure you leave your SSH port open!

Lastly, open up this file:

sudo nano /etc/default/apf-firewall

And set it to:

RUN="yes"

Now you can start up APF and you’re all protected:

sudo /etc/init.d/apf-firewall start

Setup UFW on Ubuntu 11.04

2011-06-05T00:00:00+01:00

Not much more than a few hours have passed since I posted my APF setup howto, but I’ve found something even simpler.

ufw - uncomplicated firewall. It does exactly as it says on the tin.

sudo aptitude install ufw
sudo ufw allow 22
sudo ufw enable

That’s ssh sorted, and the firewall is enabled. It’s important to allow port 22 first before enabling, otherwise you’ll get locked out. You can see the status of the firewall by running:

sudo ufw status verbose

And to allow Apache traffic:

sudo ufw allow 80

That’s it. Outgoing traffic is allowed by default. Everything else is blocked. Also, all of these rules are saved transparently, so you don’t have to worry about things like that. Reboot away, and everything will just work.

If you need to disable ufw just run:

sudo ufw disable

PHP development on Mac OS X 10.6 Snow Leopard

2011-05-27T00:00:00+01:00

Having a reliable and easy to use developer environment on my Mac is essential for my work, and the odd project I work on.

I original used the Entropy PHP packages as they were so easy to install, and just worked. Unfortunately, Marc Liyanage had stopped updating the packages with new OS X releases, so I then tried alternatives such as Macports, and more recently MAMP. However, it hasn’t been working out of late, so I went in search of alternatives.

For a long shot, I visited Marc’s PHP page and noticed that he is now endorsing a project which forked off from his own. Like his packages, it uses the default Apache install that comes with OS X, but uses their own alternative PHP package. It installs into /usr/local so that it doesn’t interfere with anything else. So here are some steps to get this all up and running.

Setup

Before I started, I uninstalled Macports and MAMP. Not required, but I didn’t want any of those packages to interfere or conflict with the new install of PHP. I now use homebrew as a replacement to Macports for tools that I require (such as MongoDB and MySQL).

Install PHP for OS X

Install PHP via http://php-osx.liip.ch:

curl -s http://php-osx.liip.ch/install.sh | bash -

It’ll ask for your password so that it can install to /usr/local. If you’re cautious of running a shell script from a webpage–as you should be!–why not view the source in your browser first to be sure all is as it should.

Setup path

If you want to use any of the PHP tools via the command line, such as pear or pecl, it’s best if you add the new PHP bin to your path. I updated mine to:

export PATH="/usr/local/php5/bin:/usr/local/bin:$PATH"

The important part here is the inclusion of /usr/local/php5/bin. You’ll then want to open a new terminal window so that the path update takes effect. Check by running:

echo $PATH

If you’re going to use pear or pecl, it may be worth checking that it’s configured to use the correct php.ini file.

Run the following command:

pear config-get php_ini

If it doesn’t say:

/usr/local/php5/lib/php.ini

Then run:

sudo pear config-set php_ini /usr/local/php5/lib/php.ini

And then just check that pecl has the same config set:

pecl config-get php_ini

That should now be PHP setup and running.

Easy Apache configuration

With some great pointers by Roger Johansson here is a great way to make configuring Apache easy for projects you work on. You’ll need to edit two files. One will make some hostname aliases to your local machine so that you can access a project via a url such as http://myproject.local. The other file is the Apache configuration file for your user, where you specify the server name and document root location. Here we go:

Edit /etc/apache2/users/yourusername.conf substituting yourusername with the actual username you use on your Mac. Update it so that it looks like this (again updating the path with your actual username):

NameVirtualHost *:80

<Directory "/Users/yourusername/Sites/">
    Options Indexes MultiViews FollowSymLinks Includes
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

<VirtualHost *:80>
    ServerName myproject.local
    DocumentRoot /Users/yourusername/Sites/myproject.local
</VirtualHost>

Now edit /etc/hosts and add to the bottom:

127.0.0.1    myproject.local

Now reload Apache, and you should be all set:

sudo apachectl graceful

You’re all set now! Hopefully this will make PHP development easier for you, as it has done for me.

Merging MKV files together

2011-05-21T00:00:00+01:00

mkvmerge -o merged_file.mkv file1.mkv + file2.mkv

Mod rewrite rules in htaccess files

2011-02-08T00:00:00+00:00

Rewrite rules in an htaccess file don’t need to leading slash, otherwise they won’t work.

Be warned. Don’t waste two hours wondering why it isn’t working.

Authentication with CodeIgniter 2.0

2011-01-29T00:00:00+00:00

I’ve started using CodeIgniter again recently, and have noticed that my how-to on implementing basic authentication could do with an upgrade for version 2.0.

The process is identical. We subclass the base CI_Controller class. This simply checks whether a logged in session is present. If not, it redirects the user to a sessions controller that can handle the logging in of a user. Once the user is logged in, the controller acts as normal.

So, first up, lets create our basic subclass:

<?php

class MY_Controller extends CI_Controller
{
    function __construct()
    {
        parent::__construct();

        $this->load->library('session');

        if (!$this->session->userdata('loggedin'))
        {
            redirect('sessions/login');
        }
    }
}

Now, this needs to be saved to application/core. This is important. You can’t add it to application/library like with the previous how-to. There have been some changes to CI that require the change. This new class is also auto-loaded automatically.

Now, the sessions controller needs to extend CI_Controller. Users don’t need to be authenticated to access this controller.

<?php

class Sessions extends CI_Controller
{
    function __construct()
    {
        parent::__construct();

        $this->load->library('session');
    }

    function login()
    {
        $this->load->view('header');
        $this->load->view('login');
        $this->load->view('footer');
    }

    function authenticate()
    {
        $this->load->model('user', '', true);

        $user = $this->input->post('user');

        if ($this->user->authenticate($user['email'], $user['password']))
        {
            $this->session->set_userdata('loggedin', true);
        }

        redirect('/');
    }

    function logout()
    {
        $this->session->unset_userdata('loggedin');

        redirect('/');
    }
}

Your login view will look something like this:

<h2>Login</h2>
<?php echo form_open('sessions/authenticate'); ?>
    <dl>
        <dt><?php echo form_label('Email', 'user_email'); ?></dt>
        <dd><?php echo form_input(array(
            'name' => 'user[email]',
            'id' => 'user_email'
        )); ?></dd>

        <dt><?php echo form_label('Password', 'user_password'); ?></dt>
        <dd><?php echo form_password(array(
            'name' => 'user[password]',
            'id' => 'user_password'
        )); ?></dd>
    </dl>
    <ul>
        <li><?php echo form_submit('submit', 'Login'); ?></li>
        <li><a href="<?php echo site_url('/'); ?>">Cancel</a></li>
    </ul>
<?php echo form_close(); ?>

Here’s a basic user model you could use. It includes hashing with extra salt security:

<?php

class User extends CI_Model
{
    public function authenticate($email, $password)
    {
        // get salt

        $salt = $this->db->select('salt')->get_where('users', array('email' => $email))->row()->salt;

        if ($salt)
        {
            // hash password with salt and find user

            $hash = sha1($salt.sha1($salt.$password));

            $user = $this->db->select('id')->get_where('users', array(
                'email' => $email,
                'hash' => $hash
            ))->row();

            return $user;
        }

        return false;
    }
}

Now all that’s left is to change the parent classes of the controllers that we want protected from CI_Controller to MY_Controller.

<?php

class Admin extends MY_Controller
{
    ...

Authentication is now in place.

A simple chat room with WebSockets

2010-11-21T00:00:00+00:00

Sorry, I promised a blog post, but instead you get a link to bitbucket with the source of a basic chat app written using WebSockets.

https://bitbucket.org/davidwinter/websockets-chat/src

Setting up SSH host shortnames

2010-08-22T00:00:00+01:00

Here’s an example setup to create SSH host shortnames. On you local computer, add the following to ~/.ssh/config:

Host server1
HostName server1.internet.com
User david

Host server2
HostName server2.internet.com
User david

Host *
User davidwinter

Now with this file saved, you can ssh into server2.internet.com with just the following command:

ssh server2

This will save you having to type out:

ssh david@server2.internet.com

And if you have public key authentication setup, it makes the process even smoother. Adding the wildcard host record at the bottom allows you to specify a default username to use for other servers to the ones you’ve not specified above.

SSH and public key authentication

2010-08-22T00:00:00+01:00

Fed up with having to type your password in each time you log into a server over SSH? Me too. Down with passwords, and in with public key authentication!

First thing to do, is to check that you have a SSH key setup already on your local computer:

cd ~/.ssh
ls

If you see some files in there starting with id_, like id_rsa.pub or id_tsa.pub, you’re all set. If not, you’ll need to generate these files:

ssh-keygen -t rsa

At the prompts you see, you can just hit enter to accept the defaults.

With your public key now in place, we need to transfer that to the server we want to log into.

scp ~/.ssh/id_rsa.pub username@remote.server.com:

Don’t forget the colon at the end. This has now transfered the public key over. Now you need to log into the remote server (with your password - last time, I promise):

ssh username@remote.server.com

Now, check if there is a .ssh directory on the remote server in your home directory:

cd ~/.ssh/

If there isn’t:

mkdir ~/.ssh

Now move the public key to a file named authorized_keys:

mv ~/id_rsa.pub ~/.ssh/authorized_keys

Now we need to set the correct permissions to ensure no one can tamper with these files:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

And you’re done. Now you can logout, and log back in again - and all going well (if your SSH server has been setup correctly - by default it usually is), you won’t be prompted for a password.

If it doesn’t work, one problem I encountered was where the actual home directory permissions weren’t set to 700. So try:

chmod 700 /home/username

All done.

Using Capistrano to deploy Mercurial changes

2010-08-22T00:00:00+01:00

We have a few production servers at work, and we have a central bitbucket repository to store our core code. Once we make a change on our testing server (!), we used to have to commit, push changes to bitbucket, and then ssh into each server, then pull changes, and update each repository. A pain in the backside! Enter Capistrano. A ruby ssh automation tool. In a few simple steps you can create a recipe file that will let you do this all with one command.

On the machine where you’d like to kick this all off—where you’ll be pushing the changes from—install Capistrano (requires Ruby and RubyGems to be installed):

sudo gem install capistrano

Capistrano is opinionated software; meaning it makes a few assumptions about your server setup which you’ll need to comply with. Mainly that either you have public key authentication setup to connect to these remote servers, or, you use the same password across all of them.

Now, create a capfile, perhaps in your home directory, and here is an example that you can modify for your own setup:

role :production_servers, "srv1.internet.com", "srv2.internet.com"

task :push_deploy, :roles => :production_servers do
    system "cd /local/path/to/repos && hg push"
    run "cd /remote/path/to/repos && hg pull && hg update"
end

With that now saved, you can issue this task with the following command:

cap push_deploy

This will first of all run a local command to push the changes to bitbucket. After that is complete, it’ll connect to each of the production servers listed, pull changes and then update the repository. And that’s it. You don’t need to do anything else. Piece of cake.

Using o2 PAYG mobile broadband

2010-06-19T00:00:00+01:00

Our internet connection via our landline has been dead since Tuesday afternoon, so I’ve needed an alternative connection in the meantime to give me my twitter fix!

I’d kept an eye on the o2 pay-as-you-go mobile broadband for a while, because it offered the cheapest, noncommittal setup. A one off £20 for the USB dongle, and then as little as £2 for 500MB for a 24 hour period.

So I plug in the dongle and we get a nice little installer to set everything up; installing drivers and the o2 Mobile Connect application. The annoying thing is that if you install going this route, you can only connect if you have this app open. You can’t just use system preferences. Also, turns out this ‘handy’ installer program then blocks you from getting to the installation files again, and the dongle will refuse to do anything unless Mobile Connect is open (you’ll get a red light on the dongle when it’s being evil).

So, what can you do?

Drag Mobile Connect.app to the trash from your Applications directory.
Then head into System > Library > Extensions and delete ZTEUSBCDCACMData.kext and ZTEUSBMassStorageFilter.kext
Go into Network in System Preferences and remove the devices starting with ZTEUSB.
Then restart your Mac.

Now, let’s install things a different route giving everyone more flexibility.

Plug in the dongle and when the CD Icon appears on the desktop, open it up, but rather than double clicking on the installer, right click on it and select ‘Show Package Contents’.
Now head to Contents then Resources. Now double click on the drv.pkg file and install this only.
Once complete, restart your Mac again.

And now to configure the connection:

Open System Preferences, and go to the Network pane.
You should have three new items in there. Click on ZTEUSBModem.
Set the telephone number to *99#
Username to o2bb
Password to password
Click on Advanced. Select Vendor; Generic, Model; GPRS (GSM/3G) and set the APN to m-bb.o2.co.uk
Click on OK and then Apply.
Now you can click ‘Connect’ and all should work fine.

That is of course, besides the crappy image compression and caching o2 force onto you! Need to find a solution for that next.

Update: Here’s a solution for getting around the image compression, and Javascript injection; use a SSH SOCKS proxy. Works a treat.

Open up Terminal.app
Run ssh -ND 9999 username@yourserver.com
Enter your password if/when prompted (you may have key authentication setup)
That’s the SOCKS proxy setup. Now go to System Preferences and open the network connection.
Click on ZTEUSBModem.
Then Advanced, Proxies
Check the SOCKS Proxy and enter in localhost and 9999
Click OK and then Apply.

All done. You should now get perfect images and no longer have Javascript inserted into all webpages. Both of which weren’t clearly mentioned when purchasing my dongle.

Redirecting Apache traffic to a maintenance page

2010-05-03T00:00:00+01:00

Here’s a simple solution to redirect users to a maintenance page in Apache. This rewrite rule can stay in your config (<VirtualHost> or .htaccess) all the time; all that you need to enable it is to create the file maintenance.html.

The first rewrite condition checks to see if the file exists, and only if it does, will it redirect all traffic to it. The second rewrite condition is there to prevent an infinite loop, by only redirecting traffic to files other than maintenance.html.

RewriteEngine On

RewriteCond %{DOCUMENT_ROOT}/maintenance.html -f
RewriteCond %{REQUEST_FILENAME} !/maintenance.html
RewriteRule ^.*$    /maintenance.html [L]

Install Mercurial on CentOS 4

2010-04-18T00:00:00+01:00

We’ve finally moved to Mercurial at work (well, we didn’t exactly move from anywhere, but that’s another story…). Our production server is running CentOS 4, which comes installed with Python 2.3.4. Mercurial requires 2.4. No Python updates available in the yum repository. What to do?

First up, install the latest version of Python to a location that won’t interfere with the system:

wget http://www.python.org/ftp/python/2.6.5/Python-2.6.5.tgz
tar xvfz Python-2.6.5.tgz
cd Python-2.6.5/
./configure --prefix=/opt
make
sudo make install

Now we just want to test the new Python install:

/opt/bin/python -V

This should come back with Python 2.6.5.

Now we install Mercurial, but tell it to use this new install of Python, rather than the default:

cd ..
wget http://mercurial.selenic.com/release/mercurial-1.5.1.tar.gz
tar xvfz mercurial-1.5.1.tar.gz
cd mercurial-1.5.1/
sudo make install PYTHON=/opt/bin/python PREFIX=/opt

Now to test Mercurial has installed successfully:

/opt/bin/hg --version

This should print out somewhere the version 1.5.1. Now Mercurial is setup, add it to /usr/local/bin so that it’s available in the path:

sudo ln -s /opt/bin/hg /usr/local/bin/hg

That wasn’t too painful was it?

Apache, PHP and MongoDB on Mac OS X 10.6 Snow Leopard

2010-01-02T00:00:00+00:00

MongoDB (from “humongous”) is a scalable, high-performance, open source, schema-free, document-oriented database.

There’s a lot of buzz brewing about it, so I wanted to give it a try with PHP on my development Mac. The following is how I went about installing Apache, PHP and MongoDB on Snow Leopard. You must have installed the Xcode developer tools (found on the Snow Leopard install DVD) and MacPorts for this to all work.

First, let’s install Apache:

sudo port install apache2

Nice and simple. Now we can move onto PHP:

sudo port install php5 +apache2 +pear

To enable PHP for Apache:

cd /opt/local/apache2/modules
sudo /opt/local/apache2/bin/apxs -a -e -n "php5" libphp5.so

I like to make some alias commands for starting, restarting and stopping Apache:

mate ~/.bashrc

alias apache-start="sudo /opt/local/apache2/bin/apachectl start"
alias apache-restart="sudo /opt/local/apache2/bin/apachectl restart"
alias apache-stop="sudo /opt/local/apache2/bin/apachectl stop"

source ~/.bashrc

Now we can make a few changes to the Apache config so that we can use our Sites directory:

sudo mate /opt/local/apache2/conf/httpd.conf

I’m using Textmate here (mate), but you can use any editor you wish, like nano.

Change the DocumentRoot:

DocumentRoot "/Users/davidwinter/Sites"

Then update the <Directory> path to the updated DocumentRoot path.

<Directory "/Users/davidwinter/Sites">

We also want to enable index.php as a directory index file:

<IfModule dir_module>
    DirectoryIndex index.html index.php
</IfModule>

And then add the PHP mime type:

<IfModule mime_module>
    ...
    AddType application/x-httpd-php .php
    AddType application/x-httpd-php-source .phps
</IfModule>

Now we need to setup the php.ini file:

sudo cp /opt/local/etc/php5/php.ini-development /opt/local/etc/php5/php.ini

sudo mate /opt/local/etc/php5/php.ini

My timezone is for London:

date.timezone = Europe/London

Now we can get onto installing MongoDB (this will install the latest stable version as of time of writing - 1.2.1):

sudo port install mongodb

And then the PHP MongoDB extension:

sudo pecl install mongo

You’ll need to add the extension to the bottom of your php.ini file:

sudo mate /opt/local/etc/php5/php.ini

extension=mongo.so

Now startup Apache using the alias created earlier:

apache-start

If you create a file in your Sites directory, called test.php, and add the following to it:

<?php phpinfo(); ?>

Then visit the page in your browser; http://localhost/test.php

You can then scroll down the page and you should see a section titled ‘mongo’. This means the extension is working!

Now we need to startup MongoDB:

mkdir -p ~/data/db
mongod --dbpath ~/data/db/

Almost there, now a simple MongoDB test in PHP:

<?php
// Connect:
$connection = new Mongo();
// Select database:
$db = $connection->my_db;
// Select collection:
$films = $db->films;

$frwl = array(
    'title' => 'From Russia With Love',
    'year' => 1963,
    'actor' => 'Sean Connery'
);
// Save array to collection:
$films->insert($frwl);

$gf = array(
    'title' => 'Goldfinger',
    'year' => 1964,
    'actor' => 'Sean Connery',
    'girl' => 'Pussy Galore'
);

$films->insert($gf);

// Count documents in collection:
if ($films->count()): ?>
<table>
    <tr>
        <th>Title</th>
        <th>Year</th>
        <th>Actor</th>
        <th>Girl</th>
    </tr>
    <?php foreach ($films->find() as $film): ?>
    <tr>
        <td><?php echo $film['title']; ?></td>
        <td><?php echo $film['year']; ?></td>
        <td><?php echo $film['actor']; ?></td>
        <td><?php echo $film['girl']; ?></td>
    </tr>
    <?php endforeach; ?>
</table>
<?php endif;
?>

Very exciting stuff. More to follow.

Remove startup scripts on Ubuntu

2009-10-20T00:00:00+01:00

sudo update-rc.d -f script_name remove

Convert git repository to mercurial

2009-10-17T00:00:00+01:00

Ensure that the mercurial convert extension is enabled:

nano ~/.hgrc

Inside that file add:

[extensions]
hgext.convert=

Save. Now do:

hg convert my-git-repo

This will create a new directory called my-git-repo-hg. This will appear empty at first, so do this:

cd my-git-repo-hg
hg checkout

All of your files will appear and you’re ready to go.

Enable PHP error logging

2009-07-14T00:00:00+01:00

In php.ini:

display_errors = Off
log_errors = On
error_log = /var/log/php-errors.log

Make the log file, and writable by www-data:

sudo touch /var/log/php-errors.log
sudo chown www-data:www-data /var/log/php-errors.log

PHP and nginx on Ubuntu: the easy way

2009-06-13T00:00:00+01:00

I’ve now changed my slice from running lighttpd to nginx. Here’s the simplest way, in around 6 commands, to get PHP up and running via FastCGI.

Install and setup

Install PHP 5:

sudo aptitude install php5-cgi

Install nginx:

sudo aptitude install nginx

Create PHP 5 FastCGI start-up script:

sudo nano /etc/init.d/php-fastcgi

Inside, put:

#!/bin/bash
BIND=127.0.0.1:9000
USER=www-data
PHP_FCGI_CHILDREN=15
PHP_FCGI_MAX_REQUESTS=1000

PHP_CGI=/usr/bin/php-cgi
PHP_CGI_NAME=`basename $PHP_CGI`
PHP_CGI_ARGS="- USER=$USER PATH=/usr/bin PHP_FCGI_CHILDREN=$PHP_FCGI_CHILDREN PHP_FCGI_MAX_REQUESTS=$PHP_FCGI_MAX_REQUESTS $PHP_CGI -b $BIND"
RETVAL=0

start() {
      echo -n "Starting PHP FastCGI: "
      start-stop-daemon --quiet --start --background --chuid "$USER" --exec /usr/bin/env -- $PHP_CGI_ARGS
      RETVAL=$?
      echo "$PHP_CGI_NAME."
}
stop() {
      echo -n "Stopping PHP FastCGI: "
      killall -q -w -u $USER $PHP_CGI
      RETVAL=$?
      echo "$PHP_CGI_NAME."
}

case "$1" in
    start)
      start
  ;;
    stop)
      stop
  ;;
    restart)
      stop
      start
  ;;
    *)
      echo "Usage: php-fastcgi {start|stop|restart}"
      exit 1
  ;;
esac
exit $RETVAL

Make start-up script executable:

sudo chmod +x /etc/init.d/php-fastcgi

Launch PHP:

sudo /etc/init.d/php-fastcgi start

Launch at start-up:

sudo update-rc.d php-fastcgi defaults

That’s it. All installed and ready to go.

Test

In your server config, add the following:

location ~ \.php$ {
    fastcgi_pass    127.0.0.1:9000;
    fastcgi_index   index.php;
    fastcgi_param   SCRIPT_FILENAME /var/www/nginx-default$fastcgi_script_name;
    include         fastcgi_params;
}

Restart nginx:

sudo /etc/init.d/nginx restart

Create a file in your web root (in the example above, /var/www/nginx-default/test.php):

<?php

phpinfo();

Visit the page in your browser and you should see the standard PHP info page. And you’re done.

Source: Aberration

Authentication with CodeIgniter

2009-02-21T00:00:00+00:00

Here’s how I do some basic authentication for a controller in CodeIgniter. It basically consists of creating a new class that extends the default Controller class. You then sub-class this on any controller that requires authentication.

Create your authentication controller in system/application/libraries and call it MY_Controller.php. It’s important that you prefix the controller name with MY_, or whatever you have specified $config['subclass_prefix'] as in your configuration.

<?php

class MY_Controller extends Controller
{
    public function __construct()
    {
        parent::__construct();
        $this->load->library('session');
        if (!$this->session->userdata('loggedin'))
        {
            header('Location: /sessions/login');
        }
    }
}

This basically checks if the session data for loggedin is set to true. If not, it’ll redirect the user to the /sessions/login URL. This means we have to create a basic controller to handle the sessions.

<?php

class Sessions extends Controller
{
    public function __construct()
    {
        parent::__construct();
        $this->load->library('session');
    }

    public function login()
    {
        $this->load->view('header');
        $this->load->view('sessions/login');
        $this->load->view('footer');
    }

    public function authenticate()
    {
        $this->load->model('user', '', true);
        if ($this->user->authenticate($this->input->post('username'), $this->input->post('password')))
        {
            $this->session->set_userdata('loggedin', true);
            header('Location: /');
        }
        else
        {
            header('Location: /sessions/login');
        }
    }

    public function logout()
    {
        $this->session->unset_userdata('loggedin');
        header('Location: /');
    }
}

The login view is a basic form with fields for username and password that submits to /sessions/authenticate. This controller then loads the user model and checks that a user with the username and password exists. If so, it sets the session data for loggedin to true and redirects the user back to the default controller. If not, it takes the user back to the login page.

Then to implement the authentication in a controller, simply do:

<?php

class SecretPlace extends MY_Controller
{
...

It’s a good idea to store your session data in your database and encrypt your cookies with $config['sess_encrypt_cookie'] = TRUE; in config.php. That way people won’t be able to snoop around and try to trick your application into thinking they’re authenticated.

A simple PHP delicious REST example with pecl_http

2008-11-24T00:00:00+00:00

<?php

$username = 'your_username';
$password = 'your_password';
$url = "https://$username:$password@api.del.icio.us/v1/posts/recent";

$request = new HttpRequest($url, HTTP_METH_GET);
$response = $request->send();
print $response->getBody();
?>

You’ll probably need to view the source in your browser to see the output.

The second argument isn’t required in this call, because HttpRequest defaults to a GET request anyway, but it’s just helpful to see where you can specify the different HTTP methods. The others being, post, put and delete.

Install pecl_http for PHP

2008-11-24T00:00:00+00:00

You will probably want to ensure that curl supports https before getting underway:

curl -V

Check that the output contains https:

>: curl -V
curl 7.19.2 (i386-apple-darwin9.5.0) libcurl/7.19.2 OpenSSL/0.9.8i zlib/1.2.3
Protocols: tftp ftp telnet dict http file https ftps
Features: Largefile NTLM SSL libz

If it doesn’t:

sudo port deactivate curl
sudo port install curl +ssl

Once you have curl with https support:

sudo pecl install pecl_http

If you followed my previous howto, you’ll want to move the module to the location specified in your php.ini file:

sudo cp /opt/local/lib/php/extensions/no-debug-non-zts-20060613/http.so /opt/local/lib/php/extensions

Restart lighttpd and then you’ll be good to go.

Install PHP5, lighttpd and Imagick on Mac OS X Leopard

2008-11-22T00:00:00+00:00

It’s a little tricky getting a nice clean install of PHP5 for OS X 10.5. The packages that I always used to depend on over at Entropy don’t seem to work anymore. At least not for Leopard. A great shame.

I’m now switching to lighttpd for my development server, so have come up with the following recipe to install PHP5 and the great Imagick extension for image manipulation.

Follow these instructions at your own risk! They worked fine on my setup but I can’t guarantee they will for you.

Install Xcode.
Install MacPorts.
Now update MacPorts:
```
sudo port selfupdate
```

Install lighttpd:

sudo port install lighttpd

Install the startup item and copy the default configuration file:

sudo launchctl load -w /Library/LaunchDaemons/org.macports.lighttpd.plist
sudo cp /opt/local/etc/lighttpd/lighttpd.conf.default /opt/local/etc/lighttpd/lighttpd.conf

Now we install PHP with the FastCGI, MySQL and Pear variants.
```
sudo port install php5 +fastcgi +mysql5 +pear
```
In order for this to work, I had to disable the port perl5.8 by using the following command:
```
sudo port deactivate perl5.8
sudo port activate perl5
```
Then run the command again and it should succeed:
```
sudo port install php5 +fastcgi +mysql5 +pear
```

Copy over the default PHP configuration:

sudo cp /opt/local/etc/php.ini-dist /opt/local/etc/php.ini

Now open up the PHP configuration file:

sudo mate /opt/local/etc/php.ini

Add the following line to the file:

cgi.fix_pathinfo=1

Find extension_dir = "./" and replace it with:

extension_dir = "/opt/local/lib/php/extensions"
extension=imagick.so

Edit the lighttpd configuration file:

sudo mate /opt/local/etc/lighttpd/lighttpd.conf

Ensure that the following is uncommented:

fastcgi.server = ( ".php" =>
    ( "localhost" =>
        (
            "socket" => "/var/run/lighttpd/php-fastcgi.socket",
            "bin-path" => "/opt/local/bin/php-cgi"
        )
    )
)

And also make lighttpd run as the user www:

server.username = "www"
server.groupname = "www"

Add the following if you want to be able to use your Sites directory in your home directory. I also changed the default document root to the OS X default:

server.modules += ("mod_userdir")
userdir.path = "Sites"
userdir.basepath = "/Users/"
server.document-root = "/Library/WebServer/Documents"

Ensure that "mod_fastcgi", is uncommented.

Now we need to setup a few directories, and make the user www the owner for them:

sudo mkdir /var/run/lighttpd
sudo mkdir /var/log/lighttpd
sudo chown www:www /var/run/lighttpd
sudo chown www:www /var/log/lighttpd

Now install ImageMagick:
```
sudo port install imagemagick
```

Now we need to download the Imagick PHP module. I was hoping this would be as easy as sudo pecl install imagick, however I got an error for that, and so a manual build and install of the module is necessary:

curl -O http://pecl.php.net/get/imagick-2.2.1.tgz
tar xvfz imagick-2.2.1.tgz
cd imagick-2.2.1/
phpize
./configure --with-imagick=/opt/local
make
sudo make install
sudo cp /opt/local/lib/php/extensions/no-debug-non-zts-20060613/imagick.so /opt/local/lib/php/extensions

And finally, you can then run LightTPD with the following. The -D flag tells lighttpd not to run in the background–you don’t need to use it:

lighttpd -f /opt/local/etc/lighttpd/lighttpd.conf -D

You should then be able to visit a PHP file in your browser to test that it works. A simple test file would be:

<?php
$img = new Imagick('pic.jpg');
$img->rotateImage('#fff', 45);
header('Content-type: image/jpeg');
echo $img;
?>

This is just a simple Imagick rotate on an image.

lighttpd and PHP on CentOS 5

2008-06-22T00:00:00+01:00

Here’s a quick run down on getting lighttpd and php running on CentOS 5.

yum install lighttpd lighttpd-fastcgi php
mkdir /var/run/lighttpd
touch /var/run/lighttpd/php-fastcgi.socket
chown -R lighttpd:lighttpd /var/run/lighttpd/
vim /etc/php.ini

Add in php.ini the following; cgi.fix_pathinfo = 1. Save and close the file.

Now open up /etc/lighttpd/lighttpd.conf. Uncomment the following line "mod_fastcgi", from the server.modules option. Then ensure that the following is uncommented:

fastcgi.server  = ( ".php" =>
    ( "localhost" =>
        (
            "socket" => "/var/run/lighttpd/php-fastcgi.socket",
            "bin-path" => "/usr/bin/php-cgi"
        )
    )
)

Save the file. Now create a test file, /srv/www/lighttpd/index.php and just include <?php phpinfo(); ?>. Now open your favourite browser and visit http://localhost/index.php. You should be greeted with a friendly PHP page, and you’re set to go.

Mac OS X/HFS+ case-insensitive? Why?

2008-05-17T00:00:00+01:00

I just found out that HFS+, the preferred file system for Mac OS X, is case-insensitive when it comes to files and directories. I don’t understand why? It’s sure as hell causing issues with stuff I’m copying from my Ubuntu PC.

Just seems like a dumb idea …

Merge video files with mencoder

2008-05-17T00:00:00+01:00

To merge video files together with mencoder is simple:

mencoder -oac copy -ovc copy file1.avi file2.avi file3.avi -o full_movie.avi

Killing a program using a batch file in Windows

2008-05-15T00:00:00+01:00

Having got my keyboard shortcut working to launch my batch file, next was to actually make it quit the emulator program when invoked.

@echo off
taskkill /F /im Fusion.exe

This is basically the command prompt version of CTRL + ALT + DEL. /im stands for ‘Image Name’, which is basically the program executable, and is the same as what you’d see in the ‘Task Manager’ window. /F just forces the program to close.

Using shortcut keys in Windows

2008-05-15T00:00:00+01:00

You won’t find me writing about Windows too much on here I promise. I’ve been forced to use Windows XP on my arcade machine due to the lack of a decent Sega Mega Drive (Genesis for anyone in the US) emulator on Linux. That has caused other problems though, because the Fusion emulator doesn’t allow me to specify a keyboard combination to quit the program via the keyboard. You may be wondering, ‘Why not just hit Alt + F4?’ Because I’m hoping not to have a keyboard plugged in all the time, and just use the joystick and buttons (which are mapped to specific keyboard keys).

I was going to try and be clever and modify my Python program to intercept a global hot key combination, and then kill the emulator, but thankfully stumbled across the idea of using a Windows shortcut with a shortcut key (the terminology gets quite confusing). The shortcut would be linked to a batch file that would quit the program.

To cut a long story short (which involved Googling… a lot), shortcut keys can only be used when the shortcut is located in either the Start menu or on the Desktop.

When I launch the shortcut when the emulator isn’t running, it opens instantly. I’ve noticed however, roughly a 15 second delay for the shortcut to invoke if the emulator is running. I’m guessing that this may be something to do with the event buffer in Windows, and the emulator taking up a chunk of the CPU. Though I’m just guessing. If anyone knows a fix for that, I’d love to hear from you.

vim: find, comment, save and quit

2008-02-29T00:00:00+00:00

For the last month, I’ve been teaching myself vim. So here’s an explanation for anyone interested in the commands I have been using:

/fd0
I
#
<esc>
:wq

Upon opening the file, I used the / command to search for ‘fd0’ in the file. In command mode, you just type a forward slash, and then the characters to search for straight after it. Hit enter, and it’ll give you the first result.

Next up, I want to insert a comment on that line. Because searching left me a few characters from the start of the line–where I want to have the comment–I enter insert mode with a capital I. This puts the cursor at the start of the current line. I then put in my # to comment the line, hit escape to get me back to command mode and then saved and quit the file with :wq.

I’m finding vim a lot friendlier to work with than emacs. It’s not quite TextMate yet, but I’m finding it rewarding learning all these different commands.

Me TV - EyeTV for Ubuntu?

2008-02-10T00:00:00+00:00

My search for an open source, Ubuntu alternative for EyeTV on OS X is nearly over. I stumbled across ‘Me TV’ a few days ago–and I’ve left it as long before writing up a post about it so that I could have a good play with it.

Installation

For those using Gutsy, you’ll need to add the Launchpad repository to your Apt sources:

sudo nano /etc/apt/sources.list

Add the following to the bottom of the file:

# Me TV
deb http://ppa.launchpad.net/michael-lamothe/ubuntu gutsy main
deb-src http://ppa.launchpad.net/michael-lamothe/ubuntu gutsy main

Save the file, and then:

sudo apt-get update
sudo apt-get install me-tv

~~If you’re running Hardy, Me TV is in the Universal repository, so you can just run sudo apt-get install me-tv.~~ Update: After contacting Michael, the developer of Me TV, he’s said that the package in the Hardy Universe repository is out of date. For the time being, you’re better off adding the LaunchPad repository, as above.

Features

Just a reminder of the features that I’m looking for in an application to replace EyeTV:

Watch all Freeview (DVB-T) channels.
Be able to pause, rewind, and forward up to live TV.
Schedule programs to record.
See the TV listings for the channels I have (EPG)
Fullscreen or windowed viewing.

Out of all of the above, there is only one feature–‘Be able to pause, rewind, and forward to live TV’–that isn’t currently available with Me TV, though I believe it should be possible to implement it in the future with developer support.

Here’s a brief run down of those features:

Watch all Freeview (DVB-T) channels

When you first launch the app, it asks if you would like it to create a channels.conf file, and then scans for the channels it can find.

Once this small step is complete, the main window opens with one of the channels on. You’ll also notice below the EPG. You can switch channels by scrolling up and down the EPG and clicking on the channel name.

Schedule Programs to Record

This is as simple as finding the program in the EPG, clicking on Record and then accepting the recording schedule. You can also set, if you want, Me TV to repeat the schedule, say; daily, weekly, etc.

Electronic Program Guide - EPG

This can be toggled on or off by right clicking in the window. You can scan through the listings every 3 hours, and switch back to what is currently on.

Full Screened or Windowed Viewing

Simply double click in the main window. Voila.

Some Suggestions

Me TV is already very user friendly, however, there are a few small things that I think could be altered or added that would make the experience even better:

On the initial launch, rather than asking if Me TV should create a channels.conf file, instead why not just a notification such as; ‘Me TV needs to scan for channels that you can watch.’ Non-technical users may get intimidated by the question about a channels.conf file. Me TV has to scan for channels. Just let the user know that’s what it’s doing.
For the region selection, include a drop down box for the various countries, and then display the regions for that selected country below–minus the country prefix.
Ditch the ‘Found Channel’ text when scanning for channels. Just show the channel name that has been found.
Move the EPG into it’s own window. It always seems as though the TV and EPG are competing against eachother. You always have to resize the window so that you can drag the EPG sizer up so that you can get a good glance at what is on across the channels. Then, when you close the EPG, the TV window is perhaps too big for viewing, so you then have to shrink the window again.
By moving the EPG into it’s own window, it doesn’t have to be restricted by the size of the TV window that you want. Users can make it larger, and therefore, rather than sticking to the 3 hour step between programs, why not just include a full 12/24 hours. At the top of the EPG window, have a button to skip by day. You can see a lot more of what’s on in one glance with a larger window, than having to scroll around with a smaller one.
When recording a program from the EPG, after clicking the Record button, don’t show the detailed schedule window afterwards. In most cases, I would bet that the user doesn’t need to see or change the details. And if they do, they can go to the scheduler to do that. You only really need the details if you’re manually creating a recording, or want to set a repeat.
For quick channel change, perhaps include a drop down box at either the top or bottom of the TV window. This removes the dependancy on switching to the EPG to switch channels, as you currently have to.

Some of these are very small, or rather picky suggestions, but none-the-less more user friendly.

Conclusion

Overall, Me TV is an excellent application. Obviosuly a lot of time has gone into developing it. If the feature set remained as it was, I would use it from now on, though I believe there are a few things that would make it an ‘easier’ experience for the user. And the one feature missing that would make it the best ever in Linux, and, hands down, the EyeTV alternative for OS X converts to Ubuntu, would be pause, rewind and forward to live TV.

I’ll be suggesting Me TV to all that I know from now on!

DVB and Mplayer

2008-02-09T00:00:00+00:00

As a follow up to yesterday’s post, here is a quick run through of how to use Mplayer to view DVB channels.

Basically, you just need to generate a channels.conf file (see this article) and place it inside ~/.mplayer/. Then run mplayer dvb:// in a terminal window. You’ll get a video window appear, and the first channel within your channels.conf file will start playing. To switch up and down between channels, you can press h and k. There’s a bunch of other helpful keyboard commands though not all of them will work with DVB.

While using Mplayer this way isn’t very user friendly, it may be helpful for some.

Watching Freeview (DVB-T) TV with VLC Player on Ubuntu

2008-02-08T00:00:00+00:00

Watching TV on my desktop. What do I want to be able to do?

Watch all Freeview (DVB-T) channels.
Be able to pause, rewind, and forward to live TV.
Schedule programs to record.
See the TV listings for the channels I have.
Fullscreen or windowed viewing.

I was able to do all of the above using EyeTV on my Mac. However, I’m trying to make the complete switch to Ubuntu, and an open-source alternative is needed.

I initially setup MythTV. It consists of two parts, a server and a client. Multiple clients around the home (or over the internet) can connect to the server. That was a little over-the-top for my needs, but MythTV is a very popular choice for TV on Linux, and does do most of the things I want from the list above, hence my decision to use it.

Once installed, I had TV playing perfectly. Though, when I wanted to start doing some work, while still watching TV, it was a little annoying that I couldn’t find anyway of watching the TV in a window–only fullscreen. Trying to find a solution on IRC, some users said there was an option to play TV in a window, however, when I tried to enable the option numerous times, I had no luck.

At this point I thought that my journey to Ubuntu was over. I needed TV on my computer, and if I couldn’t watch it in a window while doing other things, then I’d have to keep my Mac around to do that.

I’m not too sure how I stumbled across it, but I found a web page explaining that you could use VLC Player to tune into a TV card! And since then, for the last few hours, I’ve been working on getting that working. At the moment, it only ticks off a few of the boxes, but I’m sure, with a bit more Googling, I’ll be able to get the other things functioning. With the following, I can watch all my Freeview channels, in full screen or in a window.

Setting it all up

Assuming that you have your TV card drivers installed, you can start off with:

sudo apt-get install dvb-utils vlc

That installs VLC and the DVB programs needed to scan for your channels.

Now, find the digital transmitter nearest to you by visiting DigitalUK. Enter in your postcode, and the result you get back will tell you which transmitter you’re feeding off of. If you’re not in the UK, I’m not sure how you can find your closest one–Google is your friend.

You can see a list of the UK transmitters by doing:

ls /usr/share/doc/dvb-utils/examples/scan/dvb-t/ | grep uk-

Now find the filename that matches your transmitter, and substitute uk-CrystalPalace for it.

scan /usr/share/doc/dvb-utils/examples/scan/dvb-t/uk-CrystalPalace -o zap | tee ~/channels.conf

The above command will scan for channels that your TV card can pick-up, and store the configuration in a file. This file VLC then uses to tune in.

Now open up VLC. File > Quick Open File. Browse to channels.conf and open. Live TV should now start playing. To see a list of channels to switch between, go to View > Playlist.

To speed things up when wanting to launch the TV each time, you can create a launcher. I’ll create it on the Desktop for examples sake. Right click, and select Create Launcher. Give it a name, like TV, and in the Command text box, enter vlc /path/to/channels.conf. Hit ok, and you’re done. Now double click on the launcher and VLC will launch, and you’ll have the TV back on.

That’s it! Now all that is left for me to do, is to see if there is a way I can get the other things on my list operational; pause, schedule, tv listings. I’ll report back on here with any developments I find. Tune in again next week for more exciting adventures with Ubuntu!

What TV app do you use on Ubuntu?

Line breaks in OpenOffice.org

2007-05-21T00:00:00+01:00

A really frustrating issue I had when writing my project report, was that of dealing with source code formatting.

I had created a new style for source code similar to that I use on my blog; a box with a border and coloured background. The problem I was having was when pasting source code into the document. Each line had a paragraph break between itself and the next, which meant that there was a spacing between each. To fix this, I had to insert a manual line break between each line. Luckily I found a shortcut to pull this off, which was Shift + Enter.

Does anyone know of an easier way of pasting in text and using line breaks rather than paragraph breaks?

mod_rewrite and Mac OS X Personal Web Sharing

2007-05-20T00:00:00+01:00

It was very frustrating installing a fresh Wordpress on my Mac and finding that the permalinks didn’t work. Turns out the default Apache settings for personal web sharing on OS X disable .htaccess overriding.

sudo mate /etc/httpd/users/yourusername.conf

Now you’ll want to enable Apache to follow symbolic links:

Options Indexes MultiViews FollowSymLinks

And to allow .htaccess overriding:

AllowOverride All

The file should then look like:

<Directory "/Users/yourusername/Sites/">
    Options Indexes MultiViews FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

Now in System Preferences, stop and start ‘Personal Web Sharing’. Things should work perfectly now.

Graphviz - for drawing directed graphs

2007-03-14T00:00:00+00:00

This semester at University, I’m taking ‘Compiler Design & Techniques’ as a module. The second coursework involves drawing Finite State Machines diagrams that represent regular expressions. Rather than using a word processor to draw these diagrams with a rather untidy look about them, I went in search of a tool that’d help me produce high quality drawings. I’d heard that Latex was good for mathematical diagrams, but that seemed like overkill as I only wanted the diagrams, not a whole new document syntax to learn.

I come across Graphviz. Open source and available on near enough all platforms–Mac, Linux and Windows. Using a very simple syntax, you create dot files and then using the dot tool, you can generate PNG images of your diagrams.

Here is an example dot script:

digraph my_fsm {
    label = "ab*(cd)+|b";
    rankdir = LR;
    node [shape = doublecircle]; 3 4;
    node [shape = circle];
    0 -> 1 [label = "a"];
    1 -> 1 [label = "b"];
    1 -> 2 [label = "c"];
    2 -> 3 [label = "d"];
    3 -> 2 [label = "c"];
    0 -> 4 [label = "b"];
}

Following is an explanation of the above script:

I’m declaring the graph as a directed graph with digraph.
my_fsm is just the name I’ve given the graph. This can be anything.
I add a label to the graph which holds the regular expression that I’m drawing. This isn’t required, but just helps by giving it a title.
rankdir specifies the direction of the diagram. I’m specifying ‘left to right’ with LR.
So that I can differentiate between accepting states and normal states, I’m declaring that nodes 3 and 4 will have a double circle around them.
For all other nodes, I just want to use the normal circle shape.
Lastly, I declare the nodes and their edges. Also, I add a label to each edge.

Once you save the file, you can run it through the dot program using the following command:

dot my_file.dot -Tpng -o my_fsm.png

You’ll need to check where dot was installed. With the Mac installer package it is installed to /usr/local/graphviz-2.12/bin/dot, so you’ll need to modify the above command to reflect the correct location.

A very handy tool indeed. There is a PDF guide on all the commands available. Of which, I’m next going to look for how to space the graph out a bit more as it’s a little squashed at the moment.

Hope this helps.

Switch to Static IP on Ubuntu Server

2007-01-27T00:00:00+00:00

I just bought a new Linksys router for my home network and wanted to set-up my Ubuntu Web Server with a static IP address so that I could port-forward a few things. Trouble is, with only the command line it’s a little tricky to figure out exactly how to do it. Following are the steps I used.

sudo nano /etc/network/interfaces

Inside the file, you’ll see the following line:

iface eth1 inet dhcp

We want to switch from dhcp to a static IP address. Comment or delete that line, and then add the following to the file:

iface eth1 inet static
address 192.168.1.200
netmask 255.255.255.0
gateway 192.168.1.1

This line states we want to use a static IP address.
The static IP address you want to use.
The subnet mask.
This is the IP address of my router which connects to the Internet.

Save the file and then restart the network settings:

sudo /etc/init.d/networking restart

Job done.

Unix Disk Usage

2007-01-21T00:00:00+00:00

A very handy command to see the total size of a directory on a Unix-based computer.

du -sh *

This will show the total size of files and directories in your current working directory. The s flag means to show a summary–which basically just shows the top level directory only, instead of the contents of each and every folder down the file system hierarchy. The h flag shows a human readable file size.

For my home directory, I get the following output:

404M    Desktop
 30G    Documents
6.0G    Library
 32G    Movies
 18G    Music
 11G    Pictures
2.7G    Projects
 16K    Public
166M    Sites

Mac "Hot Corners" for Ubuntu

2006-12-29T00:00:00+00:00

I’ve had a bit more time recently to play around with Ubuntu and am finding the ‘Ubuntu versions’ of apps that I’ve grown to love on Mac OS X. One of which being Hot Corners which I always use to lock my screen when moving away from it.

Brightside is the Ubuntu app to do the job.

sudo apt-get install brightside

Once installed, go to System, Preferences, Screen Actions. I then checked the “Bottom right corner” and then chose “Start screensaver”. Works a charm.

Be sure that in your Screensaver settings to check the “Lock screen when screensaver is active” option.

Festive Mac Screen Saver

2006-12-17T00:00:00+00:00

Here’s a great Mac screen saver to get you in the Christmas spirit–LotsaSnow. It generates unique snow flakes that then gently fall down from the top of your screen. It’s very customisable and it’s a Universal binary.

Change your Terminal prompt - Lost style

2006-12-06T00:00:00+00:00

Fancy a change from the default prompt you get in your Terminal? Want to mimic the Terminal prompt used in Lost? I did, and during a boring lecture at Uni, decided to update it.

I’m using zsh currently, but this will also work for a bash shell (default with Mac OS X). I’m using TextMate to edit my files, but if you’re on Linux (Ubuntu, right?), then just substitute the mate command for nano or your favourite editor.

Just to make sure that you’re in your home directory:

cd ~

Now open your shell profile. If you’re using zsh:

mate .zprofile

Or if you’re using bash:

mate .bashrc

Now add the following line:

export PS1=">: "

Save and close the file.

Now, make sure you have the right colour scheme going on.

‘Terminal’ from the main menu.
Select ‘Window Settings’
Select ‘Color’ in the drop down
Select ‘Green on Black’ from the ‘Standard Color Selections’
Click the big ‘Use Settings as Defaults’
Close the window, and you’re done

Open a new Terminal window and you should get a nice Lost ‘style’ for your prompt. Nothing too fancy–just something nice and simple.

Now, you only have 108 minutes to enter the code… so don’t waste time!

Damn. I can’t wait until the next episode.

Getting Ubuntu Dapper to dance with ATI X800 GTO

2006-10-25T00:00:00+01:00

I’ve made a brand new spare PC that I’m hoping my parents can use in their spare room/study. I refuse to install Windows, but as everyone knows, I’m a keen Ubuntu fan.

My younger brother donated me his “old” ATI Radeon X800 GTO graphics card to use in it (I want to eventually get XGL + Compiz set-up on it for all the neat eye candy). Thing is, Ubuntu and the card don’t play nice straight away. Here’s how I got them to dance.

Note: This was from a fresh install of Ubuntu Dapper.

Boot from the Ubuntu install CD and when you’re presented with the install options, select “Safe Graphics Mode”. This will let Ubuntu run using your ATI card and some safe graphics drivers - you only need this so that you can complete the installation.

When the live CD environment has loaded and you see the desktop, run the ‘Install’ program from the desktop as-per-normal. Once it’s completed it’s best to upgrade Dapper to the latest version. You can either do this from the Terminal or via the notification bubble that pops up in the menu bar:

sudo apt-get update

You’ll probably need to reboot once that’s done because a new kernel (I’m guessing) has been installed. Once you’ve rebooted, type in a Terminal window:

sudo apt-get install xorg-driver-fglrx
sudo aticonfig --initial
sudo aticonfig --overlay-type=Xv

This downloads and sets up the ATI proprietary drivers for you.

Now just restart the window environment by hitting CTRL + ALT + BACKSPACE on your keyboard. The screen will go to a normal command line login type screen for about 30 seconds or less. Then the desktop will appear again. You’re now running Ubuntu with the ATI drivers. Bravo!

This was a must for me as I have bought a a nice 19” widescreen flatscreen monitor - and unless I installed the drivers, I couldn’t get the nice big resolutions. Now I have a lovely 1440 x 900 res. Lovely jubbly.

Building Apache 2.2 from source for Ubuntu Dapper

2006-10-17T00:00:00+01:00

Two reasons you might want to do this.

You want to host a Rails application using Mongrel via Apache and mod_proxy_balancer.
You’re studying in a Website administration module for your 3rd year Software Engineering degree :)

Seriously though, if you don’t want to use mod_proxy_balancer, just do a normal apt-get install of Apache 2 and you’ll be fine. mod_proxy_balancer is only available for Apache 2.2, and currently, that’s not available from the Ubuntu repositories via apt-get.

This article only covers installing Apache 2.2 - I’ll write another one for getting Subversion and PHP working shortly afterwards.

Workspace

If you’ve not got the build-essential package installed yet:

sudo apt-get install build-essential

It’s best to keep all of the source files in a seperate directory so they don’t mess up your home directory.

cd
mkdir src
cd src

Zlib

So that Apache can compress output to browsers that support it, we’re going to install Zlib first of all:

wget http://www.zlib.net/zlib-1.2.3.tar.gz
tar xvfz zlib-1.2.3.tar.gz
cd zlib-1.2.3/
./configure --prefix=/usr/local
make
sudo make install

Apache 2.2

Now download the Apache 2.2 source files:

cd ..
wget http://apache.rmplc.co.uk/httpd/httpd-2.2.3.tar.gz

Extract and move into the directory:

tar xvfz httpd-2.2.3.tar.gz
cd httpd-2.2.3/

Now to configure the build of Apache 2.2 that we want:

./configure --prefix=/usr/local/apache2 --enable-mods-shared=all --enable-deflate --enable-proxy --enable-proxy-balancer --enable-proxy-http

Besides setting the modules we’d like installed, and the location of the install, this paramater --enable-mods-shared=all is telling Apache 2.2 to build modules so that they can be dynamically loaded when it is started. This means, we can add further modules to our Apache 2.2 install when we like - as we will do with the Subversion modules and PHP.

Once the configuration is complete:

make
sudo make install

Let’s test that it’s working:

sudo /usr/local/apache2/bin/apachectl start

Now navigate to http://localhost and you should see a message saying “It works!”.

Stop Apache:

sudo /usr/local/apache2/bin/apachectl stop

Apache at start-up

Now let’s get Apache to start at boot time automatically:

sudo cp /usr/local/apache2/bin/apachectl /etc/init.d/apachectl
sudo chmod +x /etc/init.d/apachectl

What we’re doing here is copying the Apache Control script into the start-up directory.

We just need to add a few lines to the file for it to work nicely:

sudo nano /etc/init.d/apachectl

Add the followinig, so the top of the file looks like:

#!/bin/sh
#
# chkconfig: - 85 15
# description: Apache is a web server.

Save the file.

Now we need to register it with the start-up manager:

sudo /usr/sbin/update-rc.d apachectl defaults

Securing Apache

It’s also a good idea to create a dedicate Apache system user account. It’ll make your install much more secure.

sudo adduser --system apache

Now we just need to make sure that Apache runs under this user. We do that by editting the configuration file:

sudo nano /usr/local/apache2/conf/httpd.conf

You need to find the lines that say:

User daemon
Group daemon

And change them so they look like:

User apache
Group nogroup

Save the file.

Now, let’s start Apache:

sudo /usr/local/apache2/bin/apachectl start

Now to check it’s running under the new user, apache:

ps -aux | grep httpd

If you see the word apache in there, it’s working.

Check it’s all working

Now just reboot the system and before logging in, check on another machine by visiting the servers IP in the web browser and you should see the “It works!” message. This means Apache started up correctly automatically.

Building Apache 2.2 from source. Done.

Subversion 1.4.0 from source via Apache 2.2 on Ubuntu Dapper

2006-10-17T00:00:00+01:00

This howto assumes you’ve already followed my Building Apache 2.2 from source article.

Now we’re going to install Subversion 1.4.0 from source so that it can be access via Apache with authentication.

Installing Subversion

As we’ve built Apache from source, we’ll need to do the same for Subversion in order to get the Apache 2 modules mod_dav_svn and mod_authz_svn.

wget http://subversion.tigris.org/downloads/subversion-1.4.0.tar.gz
tar xvfz subversion-1.4.0.tar.gz
cd subversion-1.4.0/
./configure --prefix=/usr/local --with-apxs=/usr/local/apache2/bin/apxs
make
sudo make install

This will also add the relevant LoadModule directives into your Apache 2 configuration for you.

Creating your repository

Now, create your Subversion repository:

svnadmin create /home/yourusername/subversion/repos

We have to make that repository owned by Apache so that it can be accessed via the web:

sudo chown -R apache /home/yourusername/subversion/repos

Authentication File

Now create a user/password file for authentication:

htpasswd -cm /home/yourusername/subversion/dav_svn.passwd davidwinter

When prompted, enter your password.

Configuring Apache

Edit your /usr/local/apache2/conf/httpd.conf file with the following placed at the end:

<Location /svn>
  DAV svn
  SVNPath /home/yourusername/subversion/repos

  AuthType Basic
  AuthName "Subversion Repository"
  AuthUserFile /home/yourusername/subversion/dav_svn.passwd
  Require valid-user
</Location>

If you want access control based on different users, add the following line after the Require valid-user line:

AuthzSVNAccessFile /home/yourusername/subversion/svn_access_control

Save the file. Start Apache. To find out what you need to put in the svn_access_control file, read my previous Subversion authentication article here. for params.

Ubuntu Dapper Web Server How-to

2006-08-09T00:00:00+01:00

It’s finally here, my updated how-to on how to get Ubuntu Dapper up and running as a home web server. Perfect for hosting those small web sites and blogs. My original Ubuntu 5.10 web server how-to is still available.

This updated version is very similar to the 5.10 how-to, however, there are a few changes required. The following changes have been made:

Setting up the Multiverse and Universe repositories
The packages to install Ruby
Setting a symbolic link for Ruby
Updating rubygems
Clearing the rubygems cache

Other than those, the rest of the how-to is the same as the Breezy how-to.

This how-to assumes that you have a clean install of Ubuntu Dapper (I’m not currently using Ubuntu Dapper Server, just the basic desktop install).

The first thing to do, is get ssh installed so that if you like, you can connect and do all of this remotely.

sudo apt-get install ssh

There are some packages that are needed that aren’t in the default repositories for apt, so you have to enable them manually. Universe and Multiverse are needed.

To enable them, open up the Synaptic Package Manager

Settings
Repositories
Add
Ensure that ‘Ubuntu 6.06 LTS’ is the selected ‘Channel’
Check ‘Community maintained (Universe)’
Check ‘Non-free (Multiverse)
Click OK and let it rescan the servers.

You can then close Synaptic. apt is now all set up ready for installing the packages we need.

Before we start installing the packages we want, do a system update:

sudo apt-get update

Ruby

All on one line:

sudo apt-get install irb libdbm-ruby libfcgi-ruby1.8 libgdbm-ruby libmysql-ruby libopenssl-ruby libruby1.8-dbg ri ruby1.8-dev

In order for Ruby to work ok, do the following:

sudo ln -s /usr/bin/ruby1.8 /usr/bin/ruby

This will add a symbolic link into your /usr/bin directory so that the command ruby works, as opposed to ruby1.8.

MySQL Server

sudo apt-get install mysql-server

During this install, a Postfix config screen will appear. Select OK and then “No configuration”.

Ruby Gems & Rails

This is the standard way of installing ruby applications.

Need to download it directly from its web site:

wget http://rubyforge.org/frs/download.php/5207/rubygems-0.8.11.tgz

Then extract it:

tar xvfz rubygems-0.8.11.tgz

Move into the extracted directory:

cd rubygems-0.8.11

Run the setu-p program:

sudo ruby1.8 setup.rb

Update all installed gems on the system:

sudo gem update --system

Take note that there are two dashes preceding system.

Now that Ruby Gems is installed, we can install Rails. However, I encountered problems with this and had to clear my rubygems cache first. Do the following:

gem env gemdir

This will output a path location which you’ll need to use for the next command. For me, it was and probably is the same for you, /usr/lib/ruby/gems/1.8

sudo rm ABOVE_PATH/source_cache

This will clear your rubygems cache. Now you shouldn’t have any problems with installing rails.

sudo gem install rails -y

Apache 2

Next is the web server I’m using. Apache 2. The following command will install Apache 2, FastCGI, FCGI, PHP 5 with MySQL functionality. PHP isn’t needed for Ruby or Rails, but I’m including it so that I can run PHP apps…

sudo apt-get install apache2 libapache2-mod-fcgid libapache2-mod-fastcgi libapache2-mod-php5 php5-mysql

Once that’s done, we’ll need to enable the Apache mods we’ll be using:

sudo a2enmod fcgid
sudo a2enmod fastcgi
sudo a2enmod rewrite

You’ll notice I’ve installed two different types of Fast CGI. That’s because at some point I want to try them both out. However, at the moment, I’ve only had success with getting FastCGI working decently.

Restart Apache to make sure all is well.

sudo /etc/init.d/apache2 restart

FastCGI

I thought FastCGI would have been installed after doing the libapache2-mod-fastcgi - but for some reason, you have to install it manually.

In order to configure and build from the source files, you need to download some basic files:

sudo apt-get install build-essential

Then, grab the FastCGI files from the web site:

wget http://fastcgi.com/dist/fcgi-2.4.0.tar.gz

Extract:

tar xvfz fcgi-2.4.0.tar.gz

Move into the directory:

cd fcgi-2.4.0

Configure the installer:

./configure

Install:

sudo make install

Now, that is all of the Apache and FastCGI files installed. Later, we’ll need to configure it all.

Webmin

Webmin is ugly, but it works, and it lets us configure Apache and the Nameserver easily.

Go to http://www.webmin.com/ and download the latest version of webmin in tar.gz format.

Once downloaded, extract it (filename may vary depending on the version you download):

tar xvfz webmin-1.260.tar.gz

Move into the directory:

cd webmin-1.260

Run the set-up program:

sudo ./setup.sh /usr/local/webmin

You can use the default settings for this, or change them if you know what you’re doing. Be sure to set a password you can remember.

At the end of the set-up you’ll be given an address to use to connect to webmin. Remember this! and go to it straight away and login.

Do the following to configure Webmin to work with Apache: 1. At the top, click on the “Servers” link. 2. Select Apache Webserver. 3. Select Module Config. 4. Set the following values: * Apache Server Root - /etc/apache2 * Path to httpd executable - /usr/sbin/apache2 * Path to httpd.conf - /etc/apache2/httpd.conf * Command to start Apache - /etc/init.d/apache2 start * Command to stop Apache - /etc/init.d/apache2 stop

Save these settings.

Nameserver

I’m no DNS expert, so this is very vague and I suggest you go read a howto on DNS… here’s one I used to complete this part.

This will be different depending on your set-ups. For me, I have my domain name pointing to my web server in America that hosts http://commanderbond.net

On the server, there, I have the DNS pointing to my Internet IP at home, here in the UK.

The nameserver I’m using is Bind.

sudo apt-get install bind9

Once that’s installed, we need to configure it in Webmin. 1. Click the “Servers” link. 2. Select BIND DNS Server. 3. A message will appear about a configuration file. Select the 3rd option “Setup as an internet name server, but use Webmin’s older root server information” 4. Create a Master Zone and enter the following details: * Domain Name - yourdomain.com * Email - you@anemail.com 5. Click on “Create”. 6. Click on the “Address” link. 7. Create a new Address record using the following: * Name - yourdomain.com. (including the trailing dot) * Address - your Internet IP address 8. Click on “Create”. 9. Go to the Module Index. 10. Click on “Apply Changes”.

Set-up your web site

You’ll need somewhere to store your web site files, so I suggest to first of all create a public_html directory in your user directory.

mkdir ~/public_html

Now we’ll create a test HTML file to check all is working so far.

nano ~/public_html/index.html

Inside it put the following:

<p>Hopefully you can see this...</p>

Save the file using Control O and then Control X to quit Nano.

Now, set the permissions of the file so it’s visible to the world:

chmod 644 ~/public_html/index.html

In Webmin, go to the Apache configuration.

Create a new Virtual Server with the following: * Document Root - home/yourusername/public_html * Server Name - yourdomain.com

Save the settings. Now restart Apache:

sudo /etc/init.d/apache2 restart

Now visit http://yourdomain.com/~yourusername/ and you should see:

Hopefully you can see this…

If so, excellent! If not, hmmm… maybe you didn’t follow everything? or I made a mistake somewhere… leave a comment below.

Take a 5 minute break!

Refreshed? Right. Moving on…

phpMyAdmin for MySQL management

Go to http://www.phpmyadmin.net/home_page/index.php and download the latest stable release of phpMyAdmin in tar.gz format.

Extract:

tar xvfz phpMyAdmin-2.7.0-pl2.tar.gz

Move the directory to some place helpful:

mv phpMyAdmin-2.7.0-pl2 ~/phpmyadmin

Move into that directory:

cd ~/phpmyadmin/

Create a new configuration file:

cp config.default.php config.inc.php

Change the authentication mode to http

gedit config.inc.php

Change this line:

$cfg['Servers'][$i]['auth_type'] = 'config';

…to this line:

$cfg['Servers'][$i]['auth_type'] = 'http';

Save the file.

Back in the command line, we need to set the root password, because by default, there isn’t one!

mysql -u root

You’ll then enter the MySQL console. Enter the following:

UPDATE mysql.user SET Password=PASSWORD('newpasswordhere') WHERE User='root';

This will set the root password. Now flush the privileges:

FLUSH PRIVILEGES;

Now exit:

exit

Now, we want phpMyAdmin accessible on the server ‘globally’. So that it doesn’t matter what address we’re using, we’ll still be able to access it.

In Webmin, under Apache Webserver: 1. Select “Default Server” 2. Aliases and Redirects 3. Under “Document directory aliases” enter /phpmyadmin/ into the first text box, and /home/davidwinter/phpmyadmin/ into the box to the right of that one.

Save, and in the Terminal restart apache:

sudo /etc/init.d/apache2 restart

Go to http://yourdomain.com/phpmyadmin/

You can do any MySQL configuration here. For me, I restored my Typo database for this blog from a backup.

FastCGI configuration with a Rails application

This part is what gets a Rails app running really fast.

You’ll need to configure the FastCGI config file:

sudo nano /etc/apache2/mods-enabled/fastcgi.conf

Here is my config file:

<IfModule mod_fastcgi.c>
  #FastCgiWrapper /usr/lib/apache2/suexec2
  FastCgiIpcDir /var/lib/apache2/fastcgi
  FastCgiConfig -maxClassProcesses 2 -maxProcesses 2 -minProcesses 2 -processSlack 2
  FastCgiServer /home/davidwinter/typo/public/dispatch.fcgi -idle-timeout 120 -processes 2 -initial-env RAILS_ENV=production
</IfModule>

The following line of the above:

FastCgiServer /home/davidwinter/typo/public/dispatch.fcgi -idle-timeout 120 -processes 2 -initial-env RAILS_ENV=production

Is important. You need to specifiy a line like this for each Rails app you’ll be using under FastCGI.

Now, in your Rails application directory check that in public/.htaccess has this line:

AddHandler fastcgi-script .fcgi

And, that in the rewrite rules section, it has this:

RewriteRule ^(.*)$ dispatch.fcgi [QSA,L]

…instead of:

RewriteRule ^(.*)$ dispatch.cgi [QSA,L]

Notice .fcgi instead of .cgi

And that’s about it!

Give Apache one final restart:

sudo /etc/init.d/apache2 restart

And you should now have everything working.

As I said, any problems, leave a comment and I’ll try and help.

Set-up Typo 4 on Ubuntu Dapper

2006-07-26T00:00:00+01:00

It was very annoying to find out that the brand spanking new installer for Typo wasn’t going to install over rubygems. The irritating part was that it was SQLite causing the problem because it’s a dependency of the Typo gem. No idea why. But I don’t want to SQLite - I’m running MySQL on Apache.

So anyway, here’s how I got it working. Thanks to sprewell on the Typo IRC channel.

Download Typo from Subversion.

svn co svn://typosphere.org/svn/typo/trunk typo

Make sure that you’re inside that directory that was downloaded from Subversion.
```
bin/typo config ~/typo web-server=external database=mysql
```
Grab the MySQL schema from db/mysql.schema.sql and manually run it on your MySQL database of choice.
Go into config/ and make sure you get the database.yml file set-up to use your database settings.
Access your Typo install in your browser and you should get a prompt asking you for your admin details.
Blog away!

Cocoa-Java and Jar Frameworks

2006-07-20T00:00:00+01:00

Note: Didn’t have time to really write this article up - maybe some time I will - but I’m posting it here just in case it comes in handy for someone.

Now that I’m finished with coursework, and just awaiting my exams to start on Wednesday (can’t wait!) - I’ve got a chance to work on a little application I’ve been meaning to write for the last year or so. I’ve just never had the opportunity to get on with it.

Most of this year, I’ve been learning to develop Java applications using Swing. However, the application I’m developing is for the Mac only. I want to be using the Cocoa-Java bridge for the time being as I can’t find an RSS parsing framework as nice as Rome, which is written in Java.

So, I want to include the .jar file that holds the Rome framework in my Xcode application. This assumes you have a Cocoa-Java application set-up in Xcode.

Project > Add To Project
Select the file
New Build Phase
Select Executable
Drag jar to new build phase.
Project > Edit Active Target
Cocoa Java Specific
Change Root Directory to ‘Contents’
Change app.jar to Resources/Java/app.jar
Add desired Jar file by adding MacOS/rome.jar

Using Tomcat Ant tasks

2006-03-21T00:00:00+00:00

I’m now really loving Ant for compiling my Java projects together. It really has solved the problem of figuring out what to update, or going through countless commands to compile each package etc.

You could be wondering why I don’t just use a fancy IDE, like Eclipse to do all this for me? Well… I’ve tried. But… I just can’t ‘like’ Eclipse. I admit, I was very impressed with it - tons of functionallity, but in the end, I just couldn’t give up using the beautiful TextMate. Eclipse felt cluttered and I was claustrophobic using it.

Pathetic excuse I know, but I need something I can get on and work with.

Anyway - enough side-tracking on talk of Eclipse.

Ant. Java compiling made easy. Bring on Tomcat extension tasks with it and that saves me another aspirin for working on my Tomcat coursework which is to develop a web based quiz. To install the Tomcat tasks for Ant on Mac OS X, you just need to copy them over:

cp /usr/local/apache-tomcat-5.5.16/server/lib/catalina-ant.jar /Developer/Java/Ant/lib

Note that this installs the Tomcat tasks to the default Developer Tools install of Ant. If you’ve installed Ant manually, you’ll need to change the paths above accordingly.

Once the .jar is copied over, in your build.xml files for your various projects, you’ll need to include the following tags:

<taskdef name="deploy" classname="org.apache.catalina.ant.DeployTask" />
<taskdef name="undeploy" classname="org.apache.catalina.ant.UndeployTask" />

These task definitions only cover deploy and undeploy tasks. To use them in your project:

<target name="undeploy">
    <undeploy url="${tomcat.manager}"
        username="${tomcat.username}"
        password="${tomcat.password}"
        path="/yourwebapp" />
</target>

Where tomcat.manager is your URL to your Tomcat manager. Usually http://localhost:8080/manager. tomcat.username and tomcat.password are self-explanatory. Then just substitute yourwebapp with the name of your tomcat application directory located inside the webapp directory. For my current coursework, this is webquiz.

The deploy task is very similar:

<target name="deploy" depends="war">
    <deploy url="${tomcat.manager}"
        username="${tomcat.username}"
        password="${tomcat.password}"
        path="/${ant.project.name}"
        update="true"
        localWar="${war.dir}/${ant.project.name}.war" />
</target>

I have another target that creates a WAR file of my entire web application. This is what I’ll use to deploy my application. The only additional attributes in the deploy task are update and localWar. update will replace an existing application if it already exists so that you have the latest working copy. localWar is the location of the WAR file you want to deploy.

Simple. Hope I’ve explained this clear enough.

As of writing this, I’m having some trouble getting this working with my iBook, which I assumed had the same set-up as my Intel iMac - of which, the above tasks are working perfectly.

Access Control for Subversion with Apache2 and Authz

2006-03-03T00:00:00+00:00

My group project at University now consists of three smaller projects that provide an overall RSS service. I want to let the guys work on these, while still letting me keep my other coursework jut accessible to me. At the moment, I just have basic http authentication set-up which isn’t so great for pulling off what I want.

Please welcome on stage the Apache2 mod, authz_svn…

If you followed my other howto, you’ll have all the pre-requisites for this.

First of all, we need to create an Access Control file.

sudo nano /etc/apache2/svn_access_control

In this file, you’ll want to put some rules. I’ll first of all go over these and then provide some examples.

Permissions

There are only two types of permission:

Read only - r - a user can check-out a copy of a project.
Read and Write - rw - a user can check-out and commit changes to a project.

Users

These are the same usernames that you have set in your password file that you created in the previous howto. You can always add more users to this file using:

sudo htpasswd2 -m /etc/apache2/dav_svn.passwd bill

When prompted, enter the password for the user.

Repository Location

You specify the above rules in certain locations for the repository. These go between square brackets.

[/]

The above will specify rules for the root of the repository.

[/wowapp/trunk]

The above will specify rules for a project named ‘wowapp’ in the trunk location.

User Groups

You can create groups of users and then use those for rules. You do this under a special heading in square brackets:

[groups]
mygroup = dave, mike

This will create a group called ‘mygroup’ which ‘dave’ and ‘mike’ belongs to.

And now for some examples.

Examples

[groups]
team = bob, bill
devteam = bob, barry, brett

[/]
@team = r
bob = rw

[/wowapp/trunk]
@team = r
@devteam = rw
brenda = rw

In this example:

Created a group team which has two members; bob and bill.
Created another group, called devteam which has three members; bob, barry, brett.
In the root of the repository, I’ve given the group team read permissions.
Also, in the root, bob has read and write permissions.
In the trunk of wowapp, the group team has read permission.
Also, the devteam group has read and write permissions.
And another user, called brenda has read and write permissions.

Once you’ve created your desired access controll file, save the changes in nano by hitting CTRL O, hit enter to save the name, then CTRL X to quit Nano.

We just need to now link this access control file with our Subversion set-up.

sudo nano /etc/apache2/mods-enabled/dav_svn.conf

Here’s the example from the previous how-to:

<Location /svn>
  DAV svn
  SVNPath /home/svn

  AuthType Basic
  AuthName "Subversion Repository"
  AuthUserFile /etc/apache2/dav_svn.passwd
  Require valid-user
</Location>

All you need to add is the following line:

AuthzSVNAccessFile /etc/apache2/svn_access_control

So that the file looks like this:

<Location /svn>
  DAV svn
  SVNPath /home/svn

  AuthType Basic
  AuthName "Subversion Repository"
  AuthUserFile /etc/apache2/dav_svn.passwd

  AuthzSVNAccessFile /etc/apache2/svn_access_control

  Require valid-user
</Location>

Save the file, and then restart Apache2:

sudo /etc/init.d/apache2 restart

You should now have access control working for Subversion over Apache2.

Tomcat on Mac OS X Tiger

2006-02-20T00:00:00+00:00

Download the ‘Core’ .tar.gz version of Tomcat (as of this writing 5.5.15) from http://tomcat.apache.org/download-55.cgi. Extract and move the folder:

sudo mv ~/Desktop/apache-tomcat-5.5.15 /usr/local/

That’s all you need to do to actually install Tomcat–nice and simple. Now to create some start and stop scripts…

cd /usr/local/bin
sudo nano start_tomcat

Enter the following and save:

#!/bin/sh
export CATALINA_HOME=/usr/local/apache-tomcat-5.5.15
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home
$CATALINA_HOME/bin/startup.sh

Now a stop script:

sudo nano stop_tomcat

Enter the following:

#!/bin/sh
export CATALINA_HOME=/usr/local/apache-tomcat-5.5.15
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home
$CATALINA_HOME/bin/shutdown.sh

Voila! Now to start Tomcat, use /usr/local/bin/start_tomcat and to stop it use /usr/local/bin/stop_tomcat.

If you have /usr/local/bin included in your PATH environment variable, then you can start and stop Tomcat using start_tomcat and stop_tomcat respectively straight from the Terminal.

When Tomcat is started, you can see the installation by going to http://localhost:8080/. In order to manage Tomcat, you’ll need to modify the tomcat-users.xml file.

cd /usr/local/apache-tomcat-5.5.15/conf
sudo nano tomcat-users.xml

Inside this file, modify it so it includes the following tags:

<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
  <role rolename="manager"/>
  <role rolename="admin"/>
  <user username="davidwinter" password="123" roles="admin,manager"/>
</tomcat-users>

Where davidwinter is the username you want to use, and 123 being the password you desire.

You can then manage your Tomcat applications using this URL: http://localhost:8080/manager/html

Mac friendly Java apps in a few lines

2006-02-18T00:00:00+00:00

Mac menu-bar? Set this before you initiate your JFrame in your main class:

System.setProperty("apple.laf.useScreenMenuBar", "true");

Other systems will just ignore this and the menu-bar will appear as normal for that system.

Command keyboard button as default for keyboard shortcuts?

yourJMenuItemObject.setAccelerator(
    KeyStroke.getKeyStroke( java.awt.event.KeyEvent.VK_O,
        Toolkit.getDefaultToolkit().getMenuShortcutKeyMask()));

This will get the default key for the operating system the Java app is running on. So that means ‘Control’ for Windows and ‘Command’ for Mac.

With those small things, you can make a Java app feel like a true part of the Mac family without sacrificing it working on other systems or the need for a separate binary.

Subversion over Apache 2 on Ubuntu

2006-02-16T00:00:00+00:00

If your one of my regular readers (ha!), then you’ll know I’m starting a Group Project for University. We have 7 members in the group and without some sort of version control - managing the code we’re about to produce would be hell! So I’m setting up Subversion on my home server as a repository. Following are the steps I used to set-up Subversion over Apache 2 on my Ubuntu server.

I’m assuming you have Apache 2 already set-up. The only extra packages to download and install are subversion and libapache2-svn.

sudo apt-get install subversion libapache2-svn

This will download and install Subversion and the SVN module for Apache 2. The module itself uses WebDAV to transmit files between Subversion - so this means everything can go across port 80 without the hassle of having to worry about a firewall.

The install should automatically enable the module, but just to check:

sudo a2enmod dav_svn

It should come up saying it’s already enabled. If not, it will enable it for you.

You’ll need to configure Apache now:

sudo nano /etc/apache2/mods-enabled/dav_svn.conf

Edit the file to look something like this:

<Location /svn>
  DAV svn
  SVNPath /home/svn

  AuthType Basic
  AuthName "Subversion Repository"
  AuthUserFile /etc/apache2/dav_svn.passwd
  Require valid-user
</Location>

Change /home/svn to whatever the location of your repository is. If you haven’t created one yet, then do:

sudo mkdir /home/svn
sudo svnadmin create /home/svn

Now, you need to make Apache the owner of the repository:

sudo chown -R www-data /home/svn

To secure Subversion, do the following to create a password file:

sudo htpasswd2 -cm /etc/apache2/dav_svn.passwd bob

Replace bob with whatever username you want to use, and then when prompted enter a password.

Now restart Apache:

sudo /etc/init.d/apache2 restart

That should now all be set-up. You can try it by visiting your server http://you.server/svn. You should get a username/password dialog which you enter the details you created.

Using IRC and other IM apps at University

2006-02-06T00:00:00+00:00

At University, they’ve blocked ports for all IM applications, including IRC. This sucks, because IRC can actually be really helpful for asking for advice on things such as C++ or C. Also, our computing society has an IRC channel (#wmin-compsoc on Quakenet) and it was a real pain that we couldn’t communicate with members at the University.

I’d seen Douglas Bowman’s “how-to” on secure email using SSH tunnelling and thought that perhaps I could use this to get around the problem I was having. And, I’d be right in thinking that.

SSH rocks

I can’t stress how cool SSH is - it let’s me control my iMac at home while I’m at Uni, I can transfer my files backwards and forwards and now lets me connect to IRC.

On my iBook that I use at Uni, I’ve got Colloquy as my preferred IRC client. In the Terminal on my iBook I enter in:

ssh -N -L 6668:irc.quakenet.org:6667 davidwinter@xxx.xxx.xxx.xxx

Where xxx.xxx.xxx.xxx is the IP address of my home computer.

Once you’ve entered this command, entered your password for the SSH connection - leave the Terminal window open. When you’re finished, hit Control C to quit SSH.

In Colloquy, I create a new connection with the following details: * Chat Server: localhost * Chat Server Port :6668

Connect using those details, and then you should be able to join #wmin-compsoc perfectly.

Problem solved. I’ll try briefly to explain what the ssh command does. 1. First of all, we want a ssh connection, but without a shell prompt of the computer we’re connecting to. That’s what the -N flag is for. 2. -L flags this as a local port tunnel. 3. You then first specify the port that you’ll use on the computer your using (in my case, my iBook) to tunnel out of the network you’re currently connected to. 4. Then the server/IP address you want to access. 5. Then the IP address that you’d normally connect to if you weren’t having a firewall issue. For IRC, this is 6667. 6. Then you follow all of the above with the SSH connection, which in my case is to my iMac at home. The first part being my username on the computer, and then the IP address.

Basically, my iMac at home is being used as a bridge between Quakenet and the network at University. The whole connection is secure as it’s being encrypted over ssh. So, there’s no vulnerability for the University’s network technicians to worry about.

Obviously for this all to work, you need some kind of ‘bridge’ computer online that you can connect to. Otherwise this isn’t going to work.

Also, this could be used for getting around other kinds of firewall blocked applications. However, remember the connection will be significantly slower as first of all, it’s all being encrypted, and secondly because it’s having to go via a bridge computer.

Meebo!

As I said, I could use ssh port tunnelling to connect to MSN, iChat etc. But why bother when I can use the very cool Meebo. It’s a Web 2.0 application that let’s you connect to MSN, AIM, Yahoo! and Jabber instant messaging servers all at the same time, and simply through a web site. No need to download any software. Excellent.

Ubuntu 5.10 web server howto

2006-02-05T00:00:00+00:00

I reinstalled my home server today (well, I actually started yesterday, but today I finished from scratch again) and wrote down what I did. My aim was to have a web server set up using Apache that I could use to host this blog which uses Ruby on Rails. Also, I wanted to set-up Webmin and Bind so I could manage the DNS stuff from here.

So, here we go! If you find any errors, or have any problems, leave a comment and I’ll try and help.

The headache begins

I’m assuming that you’ve just completed a new default install of Ubuntu 5.10.

As my space PC/server is up in the loft, and it’s freezing up there, the first thing I did was to enable XDMCP on Ubuntu so I could connect to it from my Mac downstairs in my nice warm bedroom. You can read how to do that here.

With XDMCP now running, before I can connect to it via my Mac, I set the IP address of the machine to 192.168.0.1 - a nice number that’s easy to remember.

Back in my room I’m connected to Ubuntu via XDMCP and it’s as if I’m sitting up in the loft.

If you’re not using XDMCP (and even if you are you should follow this step), you’ll want to connect via SSH. Install it using:

sudo apt-get install ssh

There are some packages that are needed that aren’t in the default repositories for apt, so you have to enable them manually. Universe and Multiverse are needed.

To enable the Universe servers, I did

sudo nano /etc/apt/sources.list

Inside that file, uncomment the two lines that look something like this:

deb http://gb.archive.ubuntu.com/ubuntu breezy universe
deb-src http://gb.archive.ubuntu.com/ubuntu breezy universe

Hit Control O to save and press enter to use the same filename. Then press Control X to close Nano.

I could probably enable the Multiverse stuff in there too, but as I’ve not tested it like that, I’ll leave it to exactly how I did it.

Open up the Synaptic Package Manager 1. Settings 2. Repositories 3. Add 4. Check Multiverse 5. Click OK and let it rescan the servers.

You can then close Synaptic. apt is now all set up ready for installing the packages we need.

Before we start installing the packages we want, do a system update:

sudo apt-get update

Ruby

All on one line:

sudo apt-get install irb libdbm-ruby1.8 libfcgi-ruby1.8 libfcgi0 libgdbm-ruby1.8 libmysql-ruby1.8 libmysqlclient12 libopenssl-ruby1.8 libruby1.8-dbg mysql-common ri ri1.8 ruby1.8-dev ruby

MySQL Server

sudo apt-get install mysql-server

During this install, a Postfix config screen will appear. Select OK and then “No configuration”.

Ruby Gems & Rails

This is the standard way of installing ruby applications.

Need to download it directly from its web site:

wget http://rubyforge.org/frs/download.php/5207/rubygems-0.8.11.tgz

Then extract it:

tar xvfz rubygems-0.8.11.tgz

Move into the extracted directory:

cd rubygems-0.8.11

Run the setu-p program:

sudo ruby1.8 setup.rb

Update all installed gems on the system:

sudo gem update --system

Now that Ruby Gems is installed, we can install Rails.

sudo gem install rails -y

Apache 2

sudo apt-get install apache2 libapache2-mod-fcgid libapache2-mod-fastcgi libapache2-mod-php5 php5-mysql

Once that’s done, we’ll need to enable the Apache mods we’ll be using:

sudo a2enmod fcgid
sudo a2enmod fastcgi
sudo a2enmod rewrite

Restart Apache to make sure all is well.

sudo /etc/init.d/apache2 restart

FastCGI

I thought FastCGI would have been installed after doing the libapache2-mod-fastcgi - but for some reason, you have to install it manually.

In order to configure and build from the source files, you need to download some basic files:

sudo apt-get install build-essential

Then, grab the FastCGI files from the web site:

wget http://fastcgi.com/dist/fcgi-2.4.0.tar.gz

Extract:

tar xvfz fcgi-2.4.0.tar.gz

Move into the directory:

cd fcgi-2.4.0

Configure the installer:

./configure

Install:

sudo make install

Now, that is all of the Apache and FastCGI files installed. Later, we’ll need to configure it all.

Webmin

Webmin is ugly, but it works, and it lets us configure Apache and the Nameserver easily.

Go to http://www.webmin.com/ and download the latest version of webmin in tar.gz format.

Once downloaded, extract it (filename may vary depending on the version you download):

tar xvfz webmin-1.260.tar.gz

Move into the directory:

cd webmin-1.260

Run the set-up program:

sudo ./setup.sh /usr/local/webmin

You can use the default settings for this, or change them if you know what you’re doing. Be sure to set a password you can remember.

At the end of the set-up you’ll be given an address to use to connect to webmin. Remember this! and go to it straight away and login.

Save these settings.

Nameserver

I’m no DNS expert, so this is very vague and I suggest you go read a howto on DNS… here’s one I used to complete this part.

This will be different depending on your set-ups. For me, I have my domain name pointing to my web server in America that hosts http://commanderbond.net

On the server, there, I have the DNS pointing to my Internet IP at home, here in the UK.

The nameserver I’m using is Bind.

sudo apt-get install bind9

Set-up your web site

You’ll need somewhere to store your web site files, so I suggest to first of all create a public_html directory in your user directory.

mkdir ~/public_html

Now we’ll create a test HTML file to check all is working so far.

nano ~/public_html/index.html

Inside it put the following:

<p>Hopefully you can see this...</p>

Save the file using Control O and then Control X to quit Nano.

Now, set the permissions of the file so it’s visible to the world:

chmod 644 ~/public_html/index.html

In Webmin, go to the Apache configuration.

Create a new Virtual Server with the following: * Document Root - home/yourusername/public_html * Server Name - yourdomain.com

Save the settings. Now restart Apache:

sudo /etc/init.d/apache2 restart

Now visit http://yourdomain.com/~yourusername/ and you should see:

Hopefully you can see this…

If so, excellent! If not, hmmm… maybe you didn’t follow everything? or I made a mistake somewhere… leave a comment below.

Take a 5 minute break!

Refreshed? Right. Moving on…

phpMyAdmin for MySQL management

Go to http://www.phpmyadmin.net/home_page/index.php and download the latest stable release of phpMyAdmin in tar.gz format.

Extract:

tar xvfz phpMyAdmin-2.7.0-pl2.tar.gz

Move the directory to some place helpful:

mv phpMyAdmin-2.7.0-pl2 ~/phpmyadmin

Move into that directory:

cd ~/phpmyadmin/

Create a new configuration file:

cp config.default.php config.inc.php

Change the authentication mode to http

gedit config.inc.php

Change this line:

$cfg['Servers'][$i]['auth_type'] = 'config';

…to this line:

$cfg['Servers'][$i]['auth_type'] = 'http';

Save the file.

Back in the command line, we need to set the root password, because by default, there isn’t one!

mysql -u root

You’ll then enter the MySQL console. Enter the following:

UPDATE mysql.user SET Password=PASSWORD('newpasswordhere') WHERE User='root';

This will set the root password. Now flush the privileges:

FLUSH PRIVILEGES;

Now exit:

exit

Now, we want phpMyAdmin accessible on the server ‘globally’. So that it doesn’t matter what address we’re using, we’ll still be able to access it.

Save, and in the Terminal restart apache:

sudo /etc/init.d/apache2 restart

Go to http://yourdomain.com/phpmyadmin/

You can do any MySQL configuration here. For me, I restored my Typo database for this blog from a backup.

FastCGI configuration with a Rails application

This part is what gets a Rails app running really fast.

You’ll need to configure the FastCGI config file:

sudo nano /etc/apache2/mods-enabled/fastcgi.conf

Here is my config file:

<IfModule mod_fastcgi.c>
  #FastCgiWrapper /usr/lib/apache2/suexec2
  FastCgiIpcDir /var/lib/apache2/fastcgi
  FastCgiConfig -maxClassProcesses 2 -maxProcesses 2 -minProcesses 2 -processSlack 2
  FastCgiServer /home/davidwinter/typo/public/dispatch.fcgi -idle-timeout 120 -processes 2 -initial-env RAILS_ENV=production
</IfModule>

The following line of the above:

FastCgiServer /home/davidwinter/typo/public/dispatch.fcgi -idle-timeout 120 -processes 2 -initial-env RAILS_ENV=production

Is important. You need to specifiy a line like this for each Rails app you’ll be using under FastCGI.

Now, in your Rails application directory check that in public/.htaccess has this line:

AddHandler fastcgi-script .fcgi

And, that in the rewrite rules section, it has this:

RewriteRule ^(.*)$ dispatch.fcgi [QSA,L]

…instead of:

RewriteRule ^(.*)$ dispatch.cgi [QSA,L]

Notice .fcgi instead of .cgi

And that’s about it!

Give Apache one final restart:

sudo /etc/init.d/apache2 restart

And you should now have everything working.

As I said, any problems, leave a comment and I’ll try and help.

Java foreach loop

2005-12-08T00:00:00+00:00

Coming from a PHP background, it’s always annoyed me that I can’t use something as simple as a foreach() loop in my C++/Java applications.

However, with a recent lecture on the Collections API and Generics, that’s all changed as I’ve found the Java equivalent!

Vector<Employee> myVector = new Vector<Employee>();
for (Employee person : myVector)
{
    System.out.println(person.getName());
}

Which is the same as doing something nasty like:

Vector<Employee> myVector = new Vector<Employee>();
for (int i = 0; i < myVector.size(); i++)
{
    System.out.println(myVector.get(i).getName());
}

Saves a ton of time.

Setting up XDMCP for Mac

2005-12-08T00:00:00+00:00

I just finished setting up a spare PC I had with Ubuntu on it. Thing is, it’s damn noisy so I want it stuffed up in the loft out of the way. This is fine because I can remote log-in to it using VNC, right? The answer would be yes if I made sure I was actually logged into my account on the machine - but that would mean having to get a monitor/keyboard/mouse up there to do so.

Alternative? XDMCP. It’s kind of like VNC, but allows you to connect to your machine if no one is logged in. I don’t know the technical what’s going on in the background, but I’ve managed to get this working on my Mac.

Enable XDMCP on Ubuntu

First of all you’ll need to enable XDMCP access on the Ubuntu machine. This is really simple to do:

System > Administration > Login Screen Setup
Enter your administration password
Under the Security tab, Enable XDMCP
You can then Close the window

X11 on Mac OS X

When I upgraded to Tiger, I didn’t install X11 which you’ll need to pull this all off. Fear not - it’s easy to install.

Just slide in your Tiger install CD.
When it’s in, scroll down the Finder window slightly and you’ll see Optional Packages or something similar.
Run that app and install X11.

Connecting

Once X11 is installed, and XDMCP is enabled on Ubuntu, all that’s left is to actually connect.

Open Terminal.app
Enter in /usr/X11R6/bin/X -query XXX.XXX.XXX.XX where XXX.XXX.XXX.XXX is the IP of the Ubuntu machine
A new app will open named Xquartz and a few seconds after you should see the Ubuntu login screen appear.

And that’s all there is to it! You’re then able to login and work on Ubuntu as if you were actually sitting in front of it. Expose even works which is really handy.

The only problem I’ve had is that it’s sometimes hard to switch between windows. If you’re having problems, just click on the Xquartz.app icon in the dock to get Ubuntu back.

When you’re finished, log out in Ubuntu and then switch to Terminal.app in OS X and hit CONTROL + C to quit Xquartz.app.

Sorting a Vector using Collections.sort()

2005-12-08T00:00:00+00:00

Collections.sort() allows you to sort a Collections framework that is inherited from the List class. The Vector class is, so that’s great!

For demonstrations sake, I’ll be using a Vector that stores some products. I want to sort the products by price. I’ll assume you’d include setter and getter methods for each data member rather than declaring them public. But again for ease of this tutorial, I’ve just made them public.

public class Product
{
    public double price;
    public String name;

    public Product()
    {
        this.price(0.0);
        this.name("");
    }
}

We’d create a Vector of Products by using

Vector<Product> products = new Vector<Product>();

So that we can sort the vector, we need a way of telling Collections.sort() how to compare our Products. For example, how it can tell that ÃƒÆ’Ã¢â‚¬Å¡Ãƒâ€šÃ‚Â£1.99 is greater than ÃƒÆ’Ã¢â‚¬Å¡Ãƒâ€šÃ‚Â£0.99. I’ve already chosen that we want to sort by price.

In order for us to do this, we need our Product class to implement the interface Comparable. This interface contains only one method - compareTo() that must be implemented in our Product class. It’s this function that Collections.sort() uses.

So, how exactly do we do that?

public class Product implements Comparable
{
    // other functions in this class

    public int compareTo(Object o)
    {

    }
}

compareTo() will return an integer value that represents the comparison between two Product objects. -1 will represent a Product that is less than the one we’re checking against. 0 will represent an Product of equal values (in our case, that have the same price). And finally 1 will represent an Product that is more than the one we’re checking.

In order to perform the check we need to compare the prices. These are represented in our class as double. The class Double already implements the Comparable interface, so that means we know that compareTo() is a method of the Double class. So we’re going to use that to compare our prices.

public int compareTo(Object o)
{
    Product e = (Product) o;

    int result = this.price.compareTo(e.price);
}

What this is doing is first of all getting the price of the Product object. This returns a double. We then apply the compareTo() method to that returned the double with the object given in the parameter. We call e.price which returns the double of the price for the compare object.

After doing this, we get an integer returned and stored into result. We need to check what this value is.

public int compareTo(Object o)
{
    Product e = (Product) o;

    int result = this.price.compareTo(e.price);

    if (result < 0)
    {
        return -1;
    }
    else if (result == 0)
    {
        return 0;
    }
    else (result > 0)
    {
        return 1;
    }
}

If the result is less than 0, that means that the price of the current Product object is less than the one we’re checking against. 0 will mean that the prices are equal. And more than 1 means the price is more than the current Product.

So based on the result value, we return a new integer value that represents the comparison between two Products. Less than 0 if the object is less than the one being compared. 0 if they are equal, and more than if it is more than the compared object.

At this stage we’ve implemented the Comparable interface for our Product class. This now means our Product objects can be compared against one another based on their prices.

Now we actually want to sort our Vector. We’ll need to create the sort method.

In this example, I’m extending the Vector class:

public class ProductList extends Vector<Product>
{
    public ProductList()
    {
        super();
    }

    public ProductList sort()
    {
        // sort method
    }
}

This is where Collections.sort() comes in. Now that our Product objects can be compared, the Collections.sort() static method can access our vector and compare the objects in it and re-sort it accordingly.

    public ProductList sort()
    {
        ProductList sorted = (ProductList) this.clone();
        Collections.sort(sorted);

        return sorted;
    }

This will then return an updated ProductList object (which is the Vector storing our Products).

Bingo.