Deploying the Agaric way

Background

Where we come from

After journeying from Capistrano to Vlad we eventually use pure Rake tasks for our deployment. While Vlad was a great improvement it also had downsides. Lack of feedback from tasks made it hard to tell why they failed. Also the most interesting feature of those deployment tools is not required in our case: The ability to roll out to many servers at the same time.

Requirements

• Set up and update a number of testing/staging/demo sites and a live site
• Synchronize content between each of those and a local environment
• Create CSS from SASS before deployment to avoid generated CSS in the repository
• Environment specific settings.php files under version control

Build Tools

The road we have chosen is to use a build tool for deployment. It would have been great to find a build tool like Make for PHP but it seems the PHP community is not agreeing on a common approach the Drupal community could just hook up with. So there's Drush for interacting with Drupal from the command line and Drush Make which turns out to be not quite like Make because it lacks dependency processing. Having already some experience with Ruby and Rake I chose to create our deployment tasks with Rake using Drush whenever possible.

Deployment

One of the awesome features of Rake and other build tools is the ability to autogenerate tasks. This helps a lot considering our requirement of having different environments to deploy to. Deployment is a series of steps that need to be executed in a particular order: Preparing the build object requires to generate the CSS from SASS sources and picking the correct settings.php file. When the website is nicely packaged it has to be uploaded to the target environment, permissions have to be setup correctly, drush cc all has to be executed. All this can be programmed in a very convenient fashion with dependency based programming.

Content Synchronisation

We also have autogenerated tasks for syncing databases between our environments (they depend on a database backup task to be executed on the target of the process). Because it uses Drush to do the dump and import there is no overhead in terms of stored passwords on the developers machine or mysql configuration files on the remote servers. For syncing files rsync is used. Not having to remember passwords, filesystem paths and command line options makes those tasks very convenient.

How it actually works

Prerequisites

A good deal of our Rake tasks can be used in all of our projects. Hence it is convenient to have them available in a central place outside of any project's working copy. All things Drupal start with a D here at Agaric so we put it in ~/.drake. We use the Rakefile in a project's root directory to define the remote environments for deployment (and any project specific tasks).

To include our tasks in ~/.drake rake is invoked with the --rakelibdir option:

$ rake --rakelibdir ~/.drake

An alias can be defined to alleviate us from typing that all the time:

alias drake='rake --rakelibdir ~/.drake'

Remote environments for deployment are defined by adding an array to the ENVIRONMENTS hash in a project's Rakefile, consisting of a SSH connection string, e.g. [user@]example.com, a path where the web server expects Drupal's index.php and an optional tag to deploy.

ENVIRONMENTS = {
  "live" => ["user@example.com",
             "/path/to/document/root",
             "stable",
            ],
}

For some environments only tagged commits shall be deployed. Tasks to tag commits in accordance with a common pattern are being generated for each value in the TAGNAMES array which must also be defined in the Rakefile.

TAGNAMES = %w(qa stable)

Creating build artifacts

Deploying a Drupal site is relatively easy, usually not more than uploading the local working copy and putting the right settings in place; maybe changing file permissions. So in order to prepare deployment we only need to build a copy of the Drupal root directory with the correct settings.

Environment specific settings files are created by the file tasks etc/<env>.settings.php, e.g. etc/live.settings.php in the project root. Before deploying to the target server the database configuration must be entered there. Additional configuration that's specific to the environment can be put there, too. Those files are kept under version control.

In order to create the build artifact for an environment its task is invoked, e.g.:

$ rake build_live

Calling rake without any arguments invokes the build task for each environment. In case a tag is defined for a target environment the task takes care of checking out the tag first.

Deploying

For each environment there's a dedicated deploy task that uploads the build artifact runs drush updatedb and drush cc all:

$ rake deploy_live

Calling a deploy task will invoke the build task for the environment first. During the build process a file called BUILD_VERSION.txt is added to the Drupal directory. It contains a string indicating the git revision of the build. Use the BUILD_VERSION.txt bookmarklet to view the file when browsing a deployed site.

Synchronizing Content

Our rake library generates tasks for synchronizinging content between each environment using drush and bzip for databases and rsync for files. Some examples:

$ rake db_sync_live_to_test
$ rake db_sync_test_to_local
$ rake file_sync_live_to_test
$ rake file_sync_test_to_local

Walkthrough

Starting a new Drupal project

To start a new project named example, create a directory example in your local development environment and copy the starter sample rake configuration file (config.rb.example) from Agaric's Rake repository. (You should check out that Rake repository to a local folder, e.g. ~/.drake if you have not already.) Here are the steps that would be repeated for each project:

$ mkdir example
$ cd example
$ drush -y dl --drupal-project-rename=web
$ mkdir web/sites/all/modules/contrib
$ mkdir web/sites/default/files
$ chmod -R a+w web/sites/default
$ cp ~/.drake/config.rb.example config.rb
$ git init
$ git add .
$ git commit -m "Start new Drupal project"

Note: The command agc example NEW in Agaric's scripts repository automates the above, assuming ~/code/ as the home for site projects.

Configuration

The first thing to do is changing the remote environments in the Rakefile.

ENVIRONMENTS = {
  "live" => ["example@sojourner.mayfirst.org",
             "/home/members/agariclabs/sites/example.com/web",
             "stable",
            ],
  "test" => ["simone.mayfirst.org",
             "/var/local/drupal/example-test",
            ],
}

Our tasks use SSH via FileUtils::sh for executing commands on remote servers. This means all settings in .ssh/config like usernames, gateways, agent forwarding, etc. are respected.

Set up a remote repository

Having made just the basic changes you should create a repository on the test server to share and deploy from. There's a task for that:

$ rake init_repo

It creates a new bare repository on the infrastructure server and pushes the local repository to this new remote. It also takes care of adding this remote repository to the .git/config file.

Deploying to a test environment

Create a database and deploy the code to the test environment defined in config.rb:

$ rake setup_test
$ rake deploy_test

Historical note

In its former internal-only location, this document was (in)famously known as node/177