« Die, default namespaces, die | Main | WWW cubed: syndication and scale »

Setting up Cruisecontrol...

...took most of the day. It's a good tool, but finicky to setup. Also there isn't much info to be found out there on how to use it with Subversion. Here's what I threw together to port a largish modularized project over to it.

Fair Warning: At the end of the day I have no idea whether this is an idiomatic way to use Cruisecontrol - so - this howto might not be good for you. I also ought to say that I wouldn't have been able to set it up in a reasonable amount of time without having the source code to look at (and possibly, that's not the point of having the source code to look at). Once it's up and running tho' it takes care of itself.

Documentation

The documentation is... lacking. It makes you wish every open source project was like Hibernate or MySQL. Some of the attributes and element names are wrong or in camel case when they shouldn't be; some steps are left out; this is annoying but nothing you can't get past after flailing about for a bit and typing random stuff at the console until it works.

Layout

First, download Cruisecontrol and symlink it to /usr/java/cruisecontrol - then created a workspace under a user account's home folder:

    [propylon@cvsdub2 propylon]$ mkdir /home/propylon//builds
    [propylon@cvsdub2 propylon]$ mkdir /home/propylon//builds/work
    [propylon@cvsdub2 propylon]$ mkdir /home/propylon//builds/logs

work is where Cruisecontrol will find your projects; logs is where it will store log data.

    [propylon@cvsdub2 builds]$ drwxr-xr-x  3 propylon propylon  4096 Sep 10 22:23 work
    [propylon@cvsdub2 builds]$ drwxr-xr-x  3 propylon propylon  4096 Sep 10 15:00 logs

Build

Building is the easiest part of the process. Here the documentation actually works out. To build a war file for web reporting add a file called override.properties to cruisecontrol/reporting/jsp pointing the properties at the logs directory you just setup,

    user.log.dir=/home/propylon/builds/logs
    user.build.status.file=currentbuildstatus.txt
    cruise.build.artifacts.dir=/home/propylon/builds/logs

and run the build script by passing a 'war' argument to it. Drop that war into a servlet container, and you're done.

Config file (almost, first a scripting diversion)

The Cruisecontrol config is clunky. But before we get to that, let's point out that Cruisecontrol at heart wants to fire up a JVM per project. Maybe there is some drop-dead simple way around this, but I couldn't figure it out beyond patching the code itself. To allow for multiple projects, we create the following artefacts, for each project:

  • run script: this is the file you'll use to boot Cruisecontrol for the project.
  • config: this is the Cruisecontrol config file for the project
  • ant file: this is a buildfile that lives outside the project structure. It does three things:
    1. blows away the last checkout under the 'work' folder
    2. runs a full checkout of the project
    3. calls a top level ant file in the project

The run file looks like this:

    #!/bin/sh
    export ANT_HOME=/usr/java/ant
    export JAVA_HOME=/usr/java/jdk14
    # put autobuild junk into a dummy folder
    export TOMCAT_HOME=/home/propylon/builds/HOME/TOMCAT_HOME
    # if you're reading a blog entry, skip these two
    export PROPELXBI_HOME=/home/propylon/builds/work/iams
    export IAMS_HOME=/home/propylon/builds/work/iams
    export PATH=$PATH:$ANT_HOME/bin
    ccmain=/usr/java/cruisecontrol/main/bin/cruisecontrol.sh
    $ccmain -projectname iams  -configfile cc-iams-config.xml &

and the build file looks like this:

    <?xml version="1.0"?>
    <project name="cc-iams" basedir="." default="build">
      <property file="cc-svn.properties" />
      <path id="project.classpath">
        <pathelement location="${svnjavahl.jar}" />
        <pathelement location="${svnant.jar}" />
        <pathelement location="${svnClientAdapter.jar}" />
      </path>
      <taskdef resource="svntask.properties" classpathref="project.classpath"/>
      <target name="build">
        <delete dir="work/iams"/>
        <svn  username="xxxxx" password="xxxxx">
          <checkout url="http://cvsdub2/svn/iams/trunk" revision="HEAD" destPath="work/iams" />
        </svn>
        <ant antfile="build.xml" target="build" dir="work/iams"/>
      </target>
      <target name="cp" description="print the build classpath">
        <property name="cp" refid="project.classpath" />
        <echo>${cp}</echo>
    </target>

All these files go under that builds folder we just made. Here's what things look like so far:

    [propylon@cvsdub2 builds]$ -rwxr-xr-x  1 propylon propylon   393 Sep 10 21:25 cc-iams-build.sh
    [propylon@cvsdub2 builds]$ -rw-r--r--  1 propylon propylon  1599 Sep 10 22:28 cc-iams-config.xml
    [propylon@cvsdub2 builds]$ -rw-r--r--  1 propylon propylon   905 Sep 10 12:27 cc-iams.xml
    [propylon@cvsdub2 builds]$ drwxr-xr-x  3 propylon propylon  4096 Sep 10 22:23 work
    [propylon@cvsdub2 builds]$ drwxr-xr-x  3 propylon propylon  4096 Sep 10 15:00 logs

You could just call straight into the project's ant file and skip the new build file, but I like the idea of a separation between automated and project build systems (must... use... more... indirection). Incidentally, the project itself has ten or so standalone modularized builds that can be run from the master build referenced above.

Config file (almost, really, some subversion first)

It can seen from the build file above that the project is in Subversion. That means we need to install the svnant libraries. This is easy to do, just unpack the distribution into /usr/java/svnant. The cc-svn.properties file can be reused across all projects and looks like this:

    svnant.version=0.9.1
    lib.dir=/usr/java/svnant/lib
    svnjavahl.jar=${lib.dir}/svnjavahl.jar
    svnant.jar=${lib.dir}/svnant.jar
    svnClientAdapter.jar=${lib.dir}/svnClientAdapter.jar

(it's lifted directly from the example provided by svnant)

So here's where we're at:

    [propylon@cvsdub2 builds]$ -rwxr-xr-x  1 propylon propylon   393 Sep 10 21:25 cc-iams-build.sh
    [propylon@cvsdub2 builds]$ -rw-r--r--  1 propylon propylon  1599 Sep 10 22:28 cc-iams-config.xml
    [propylon@cvsdub2 builds]$ -rw-r--r--  1 propylon propylon   905 Sep 10 12:27 cc-iams.xml
    [propylon@cvsdub2 builds]$ -rw-r--r--  1 propylon propylon   608 Sep 10 14:28 cc-svn.properties
    [propylon@cvsdub2 builds]$ drwxr-xr-x  3 propylon propylon  4096 Sep 10 22:23 work
    [propylon@cvsdub2 builds]$ drwxrwxr-x  5 propylon propylon  4096 Sep 10 18:17 HOME
    [propylon@cvsdub2 builds]$ drwxr-xr-x  3 propylon propylon  4096 Sep 10 15:00 logs

Ant 1.6 (config file is next, I swear)

Cruisecontrol ships with Ant 1.5.3. I found this out when it couldn't run an ant script with an import task, but for the sake of this howto I'm going to pretend I knew that upfront. The way to get around this without changing its classpath in the startup script is to use the "antscript" attribute from the ant element to point to a script instead, ie:

    <schedule interval="21600">
      <ant  antscript="/home/propylon/builds/ant16.sh"  
          buildfile="cc-iams.xml" target="build" />
    </schedule>

The script in turn points at your own ant distribution:

    #! /bin/sh
    export ANT_HOME=/usr/java/ant
    antmain=${ANT_HOME}/bin/ant
    $antmain "$@"

let's add that to the builds folder:

    [propylon@cvsdub2 builds]$ -rwxr-xr-x  1 propylon propylon    77 Sep 10 22:18 ant16.sh
    [propylon@cvsdub2 builds]$ -rwxr-xr-x  1 propylon propylon   393 Sep 10 21:25 cc-iams-build.sh
    [propylon@cvsdub2 builds]$ -rw-r--r--  1 propylon propylon  1599 Sep 10 22:28 cc-iams-config.xml
    [propylon@cvsdub2 builds]$ -rw-r--r--  1 propylon propylon   905 Sep 10 12:27 cc-iams.xml
    [propylon@cvsdub2 builds]$ -rw-r--r--  1 propylon propylon   608 Sep 10 14:28 cc-svn.properties
    [propylon@cvsdub2 builds]$ drwxr-xr-x  3 propylon propylon  4096 Sep 10 22:23 work
    [propylon@cvsdub2 builds]$ drwxrwxr-x  5 propylon propylon  4096 Sep 10 18:17 HOME
    [propylon@cvsdub2 builds]$ drwxr-xr-x  3 propylon propylon  4096 Sep 10 15:00 logs

Config file (at last!)

Here's a basic config file:

    <cruisecontrol>
      <project name="iams" buildafterfailed="false">
        <bootstrappers>
          <currentbuildstatusbootstrapper file="logs/iams/currentbuildstatus.txt"/>
        </bootstrappers>
        <modificationset quietperiod="60">
          <svn
            LocalWorkingCopy="work/iams"
            username="xxxxx"
            password="xxxxx"></svn>
        </modificationset>
        <!-- 6 hours -->
        <schedule interval="21600">
          <ant antscript="/home/propylon/builds/ant16.sh"  
              buildfile="cc-iams.xml" target="build" />
        </schedule>
        <log dir="logs/iams" encoding="UTF-8">
        </log>
        <publishers>
          <currentbuildstatuspublisher file="logs/currentbuildstatus.txt"/>
          <htmlemail
              mailhost="mail.propylon.com"
              returnaddress="noreply-cruisecontrol-at-propylon.com"
              defaultsuffix="-at-propylon.com"
              buildresultsurl="http://cvsdub2:8080/cruisecontrol/buildresults/iams"
              css="/usr/java/cruisecontrol/reporting/jsp/css/cruisecontrol.css"
              xsldir="/usr/java/cruisecontrol/reporting/jsp/xsl"
              logdir="logs/iams"
              subjectprefix="[build-nanny] ">
            <map alias="list" address="S0070-at-724.ie"/>
            <map alias="tommy.lindberg" address="tommy.lindberg-at-propylon.com"/>
            <map alias="bill.dehora" address="bill.dehora-at-propylon.com"/>
            <always address="list"/>
            <failure address="tommy.lindberg" reportWhenFixed="true"/>
            <failure address="bill.dehora" reportWhenFixed="true"/>
          </htmlemail>
        </publishers>
      </project>
    </cruisecontrol>

The first thing to say about this is that the Subversion task's name is 'svn' not 'Subversion' (did I say the documentation was lacking?). Anyway, the above will do roughly the following:

  • Schedule a build for a project called 'iams'
  • Stop trying to build after a failure, unless there is a change in the repository (buildafterfailed="false")
  • keeping track of whether the project's being built (file="logs/iams/currentbuildstatus.txt")
  • look for projects changes via subversion (LocalWorkingCopy="work/iams")
  • Attempt to run a build every 6 hours (interval="21600")
  • Only run a build if something changed
  • Use a shell script to invoke the buildfile (antscript="/home/propylon/builds/ant16.sh" buildfile="cc-iams.xml" )
  • Log everything (dir="logs/iams")
  • Annoy people with the build results ("htmlemail"); some people will get annoyed at every build ("always"); some only when there's a problem ("failure")

By the looks of things, configuration has options for a number of other features, but this setup is fine.

Ant extensions

Cruisecontrol wasn't picking up jdepend or tomcat tasks during builds even thought these were installed into Ant's lib folder and are referenced as such by the individual build files (totally different classpaths of course, doh). This broke perfecty good builds. The run scripts (in cruisecontrol/main/bin) have a classpath variable called CRUISE_PATH which includes Ant. Dropping the jars in question into Cruisecontrol lib folder and hacking the paths onto the end of CRUISE_PATH variable in the script solved that problem (I'll have to remember to put them somewhere else or a Cruisecontrol upgrade will break the build). NB: I did this before discovering I needed to bung an Ant 1.6.x shell script into the process - pointing at another Ant might end up having things work for you without making modifications; if not, fix up the Cruisecontrol scripts.

Do one checkout

Cruisecontrol does not seem to check the project out the first time it's run; you have to do this (again maybe this is possible to setup). So cd to the work folder and:

    [propylon@cvsdub2 builds]$ cd work/
    [propylon@cvsdub2 work]$ mkdir iams
    [propylon@cvsdub2 work]$ svn co http://cvsdub2/svn/iams/trunk .

Murray Walker Moment: Go Go Go

Ok, we're done. Start up Cruisecontrol using the cc-iams-build.sh script

    [propylon@cvsdub2 builds]$ ./cc-iams-build.sh

End

Clearly, there are a number of alternate ways to do this; much of it will come down to how you like to organize specifics, ie, whether you put things like ANT_HOME in a .profile or in a script, or whether you want a separate buildfile for Cruisecontrol use. I've also left out a lot of details, like setting the executable bit on the shell scripts, testing the cc-iams.xml ant file is working, checking that user has permissions to write into certain folders and can run certain things and so on. The essential setup described here will also work on windows, once you fix up the paths and use .bat files instead.

Cruisecontrol has some nice touches: html mail, an indicator near the top mentioning (in red) that "this project doesn't have any tests", a web console, pie charts indicating the proportion of busted builds, build only if something changed, a pause if the project is being checked into, a list of changes made since the last build, a list of deployed artefacts, and test results. It seems once you have one project going, Cruisecontrol just works, and you can cut and paste the config file and various scripts for the next one. But I'm starting to understand why certain Thoughtworks and Atlassian open source bots are hacking out Damagecontrol.


September 11, 2004 01:26 AM

Comments

gab
(September 11, 2004 11:21 AM #)

maybe you should check out damagecontrol. I guess it is even more undocumented :)

Steve Loughran
(September 11, 2004 12:39 PM #)

manual trackback. we share your pain :(

jerome
(September 14, 2004 03:42 PM #)

Just to say that there's a momentun to improve documentation, so it's going to improve in the forthcoming months. The web site improved a lot in the past 2 months and now the doc contents are being reworked.

Also if you feel you have some courage, please add your notes to the official Wiki or point them from the user list if you haven't done it before.

Jerome

Trackback Pings

TrackBack URL for this entry:
http://www.dehora.net/mt/mt-tb.cgi/1414