Ever since getting serious in documenting my install process for my developer machine, I've found that while executing the steps themselves is fairly quick, it still takes me hours to get all the other applications I use on a daily basis downloaded and installed. While browsing my Twitter stream, I came across the pivotal_workstation project on GitHub. While I had some experience with automated provisioning systems like Chef and Puppet, it never occured to me that it might be the solution to this exact problem. In this blogpost I will outline all the different components involved, how to setup your computer with these tools and how to contribute to make this project even better.

Chef and soloist

Let's start at the beginning. If you want to install program X, you need some kind of recipe that describes all the steps that need to be taken to do this. For example, download the .dmg from this url, open it and copy the .app to /Applications. This is what cookbooks are for. Cookbooks are a collection of recipes that contain these instructions. I mentioned such a cookbook before, pivotal_workstation is one of these cookbooks. These recipes are written in Ruby so they should be easy to follow. More on them later.

Now the Chef application is the one that takes these recipes and executes them. The Chef application uses a json configuration file that describes what cookbooks and recipes to use. This json file can either come from a Chef server (so you can manage your cookbooks in one location and every computer or server will get the information from that location) or it can run as “Chef Solo” locally without the Chef server. This Chef Solo mode is the one I use since setting up a computer is mostly a one shot business.

There is only one issue with Chef Solo, manually crafting the json configuration is a real pain. But ofcourse there is a solution in the form of a gem called Soloist. Soloist uses a clean and easy to understand configuration file in YAML and uses this to execute and configure Chef Solo. An example soloistrc:

cookbook_paths:
- /Users/roderik/cookbooks
recipes:
- pivotal_workstation::create_var_chef_cache
- pivotal_workstation::xquartz
- pivotal_workstation::locate_on
- pivotal_workstation::1password
- pivotal_workstation::gem_setup
- pivotal_workstation::bash4
- pivotal_workstation::bash_it
- pivotal_workstation::bash_completion
- pivotal_workstation::dropbox
- pivotal_workstation::firefox
- pivotal_workstation::github_for_mac
- pivotal_workstation::bartender
- pivotal_workstation::istatmenus
- pivotal_workstation::phpstorm
- pivotal_workstation::things
- pivotal_workstation::mysql
- pivotal_workstation::prevent_time_machine_from_prompting_to_use_new_hard_drives_as_backup_volume
- roderik_workstation::sublime_packages
- roderik_workstation::inputrc

Apart from installing XCode, building these soloistrc's is the only manual work you need to do to install a fully operational developer machine. But even that part is easy since there is a soloistrc-builder webapp available.

Let's setup our brand new computer.

First off, let's install the Command Line Tools to be able to use GIT and compile stuff. You can go for either the Command Line Tools for Xcode that contain just basics, or go for the full Xcode. If you go for the full Xcode, you still need to install the Command Line Tools but it is easy to do under Preferences > Downloads.

Next, all the rest in one fell swoop:

bash <(curl -s https://raw.github.com/roderik/roderik_workstation/master/bootstrap-public.sh)

Done… Kind of an anticlimax, but hey, this is why I use this system. Let's go into some details so you understand what is going on.

The bootstrap file

I've included an extract of the bootstrap file below. It does 5 things.

1. make sure soloist and it's dependencies (like chef) are installed
2. create a cookbooks folder in your homedir
3. write a soloistrc file, it is this list that you need to build to create your own perfect environment. This list is way larger in the bootstrap file used in the oneliner above.
4. clone and pull all the cookbooks that are needed
5. run soloist

#!/bin/bash
 
if rvm --version 2>/dev/null; then
  gem install soloist
else
  sudo gem install soloist
fi
 
mkdir -p ~/cookbooks; cd ~/cookbooks
 
cat > soloistrc <<EOF
cookbook_paths:
- $PWD
recipes:
- pivotal_workstation::create_var_chef_cache
- pivotal_workstation::xquartz
- roderik_workstation::sublime_packages
EOF
 
if [[ -d pivotal_workstation ]]; then
  cd pivotal_workstation && git pull && cd ..
else
  git clone https://github.com/roderik/pivotal_workstation.git
fi
if [[ -d dmg ]]; then
  cd dmg && git pull && cd ..
else
  git clone https://github.com/opscode-cookbooks/dmg.git
fi
if [[ -d roderik_workstation ]]; then
  cd roderik_workstation && git pull && cd ..
else
  git clone https://github.com/roderik/roderik_workstation.git
fi
 
soloist

Making it your own

The pivotal_workstation cookbook is used by the employees of Pivotal and does what they want it to do. The roderik_workstation cookbook, is my own personal preference. These are not necessarily your perfect settings. To keep it manageable I suggest you fork the pivotal_workstation cookbook (like I've done and I use my own fork in the bootstrap file) and either fork roderik_workstation or create your own _workstation repo where you put your own settings. This will give you all the flexibility to contribute (more later on) and to make your own choices.

There are some useful things you might want to know when you are customising, lots of recipes have a related attributes file. These contain configurable settings that you can override in your own cookbook. For example the Sublime Text packages I want to install, by using node.override and setting my own array, it uses my config over the pivotal one.

Same goes for templates, they can also be overridden. Again with Sublime Text, I use my own recipe, that includes the pivotal_workstation one, but overrides what config file template it should use.

Contributing

Contributing is very easy, the guys at Pivotal are very responsive in approving pull requests and there are so many recipes in this cookbook that 99% of the things can be done with some creative copy pasting (I'm not a Ruby programmer, nor am I very familiar with Chef, and yet I was able to become one of the biggest contributers to this project). The only discipline you need it to never install an app without trying to write it as a recipe first.

At this point I will show some signature recipes, so you can get started immediately and hopefuly skip some trail and error steps I had to take :)

Installing from Homebrew

Just include the pivotal_workstation::homebrew recipe and use brew_install with the packagename. Easy!

include_recipe "pivotal_workstation::homebrew"
brew_install "pwgen"

Installing from DMG

Copy the alfred.rb recipe you see below, change all the references to the app, the download location, update the checksum (sha256), messages and you add it to your soloistrc/bootstrap.sh file.

Important, If you use dmg_package "Alfred" it expects the DMG to be called Alfred.dmg, the mounted volume at /Volumes/Alfred and the app to be called Alfred.app. This is not the case with Alfred, the volume is named Alfred.app so we added the volumes_dir line. Some DMG files have an EULA, you can disable this by using accept_eula true like in the OmniGraffle recipe Sometimes the dmg name is different, then you use dmg_name "googlechrome" like in the Google Chrome recipe. You won't need this for most dmg files.

dmg_package "Alfred" do
  volumes_dir "Alfred.app"
  source "http://cachefly.alfredapp.com/alfred_1.3.1_261.dmg"
  checksum "c951c4dc05ff1091359358d710142cabef2c190be41f799244669f879cff7e80"
  action :install
  owner WS_USER
end

Installing from ZIP

Copy the 1password.rb recipe you see below, change all the references to the app, the download location, update the checksum (sha256), messages and you add it to your soloistrc/bootstrap.sh file.

unless File.exists?("/Applications/1Password.app")
 
  remote_file "#{Chef::Config[:file_cache_path]}/1password.zip" do
    source "https://d13itkw33a7sus.cloudfront.net/dist/1P/mac/1Password-3.8.20.zip"
    owner WS_USER
    checksum "56aef138f06fc92d641c424742bc40887cd19e6029f20409ec06ad5514b8cff1" # optional
  end
 
  execute "unzip 1password" do
    command "unzip #{Chef::Config[:file_cache_path]}/1password.zip -d #{Chef::Config[:file_cache_path]}/"
    user WS_USER
  end
 
  execute "copy 1password to /Applications" do
    command "mv #{Chef::Config[:file_cache_path]}/1Password.app #{Regexp.escape("/Applications/1Password.app")}"
    user WS_USER
    group "admin"
  end
 
  ruby_block "test to see if 1Password.app was installed" do
    block do
      raise "1Password.app was not installed" unless File.exists?("/Applications/1Password.app")
    end
  end
 
end

Wrapping up

Using Chef to automate all these tasks for me has been a revalation for me, while i'm sure that there will be many many improvements in the next months, I've found that this system allows me te exactly recreate and configure my developer environment. But as always, some things are still missing. Triggering installs from the App Store, entering licences, configuring Mail.app etc. All things to research in the future. If I find anything of note, be sure to expect followups.