Continuous building AND testing of Eclipse Plugins

(Version: 23. 10. 2007, rkleeb)

On this page I'll sum up all the steps that I had to do to get CruiseControl running the way I wanted it to. This means:

  • Having CruiseControl building my Eclipse Plugin
  • Having CruiseControl preparing installable Plugins on a UpdateSite (nightly builds)
  • Having CruiseControl running the defined Plugin-Tests (Tests that actually start a new Instance of Eclipse, not just simple JUnit tests)

This article is based on my experiences using PluginBuilder to get a continuous Build of the groovy-eclipse Plugin for our term project at the HSR.

Feedback's of course always appreciated (rkleeb at hsr dot ch)

Step by Step

  • The very first step in a project using CI (Continuous Integration, [ PragAutomation ]) should be the installation of CruiseControl. The installation itself is pretty straightforward, just unzip the file that you downloaded and start it. I recommend to start with a simple build like a Latex Document or a very simple Java project.
  • If your CruiseControl is running fine with the simple Project it's time for the next Step. Building the Plugins in your local Eclipse-Installation. We used PluginBuilder [ PBuild ] by Markus Barchfeld (He also wrote an interesting article about this subject [ BuildAndTesAut ]). After installing the plugin you can easily follow the manual on the PluginBuilder Website.
    For starters I recommend to try the local build, meaning that the sources are copied directly from your local workspace (this is just faster than checking out the whole repository every time you wanna try out something that you quickly changed in your configuration).
  • If you run into problems, you should have a look at the Files build_local.properties and pluginbuilder.config make sure that all the variables correspond with your system and setup. Make sure that whenever you use backslashes in these files (Windows directories!) to put two of them.
  • Make sure to delete all the folders that contain compiled source files.
  • One of the problems I had was the compilation-order inside the different plugins: Since we had Java classes that extended or implemented groovy-classes we had to make sure that the groovy classes would be compiled first.
    The question was: Where would we change such a thing that's individual for one of the plugins if we can't use the build.xml in the Plugin-directory nor the global build.xml(We can't use the build.xml because PluginBuilder regenerates the build.xml Files in every run). The solution is to use the customBuildCallbacks.xml file in the plugin projects. It offers you the possibility to step into the different stages of the build process. In our special case we simply moved the Tasks that generated the Groovy classes into the "pre-compile" section.
  • If this step was successful I recommend to prepare a fresh Eclipse directory ready and copy the generated plugin from the buildDirectory into the plugins folder of Eclipse. After starting this Eclipse instance you should be able to use the features of your Eclipse plugin.
  • The next step would be, to get the building done with source files checked out from the SVN Repository. To get this done you just have to change the settings in the PluginBuilder configuration of your Project. Make sure to have a svn command line client installed on your system. (Attention Windows Users: I recommend the usage of the native SVN Client and not the Cygwin one, the Cygwin SVN Client caused some problems because it didn't recognize the "Windows-Style" Directories).
    This test should also lead to the same results as the test with the copies from the workspace. (If you run into problems with error messages that tell you that a certain version of a component is missing you should check the all.map file to make sure it contains all the necessary paths)
  • If the compilation from SVN works we're ready for the next step, the integration of the Tests:
    First of all, enable "Run Tests" in your PluginBuilder configuration. We skipped the "Regexp-part" to define the tests and edited the responsible Ant File (automatedTests/run-tests.xml) manually. The following Listing shows our Version of the two targets that are commented out in the Version that ships with PluginBuilder. We commented the original "run" as well as the "-runAutoTests" target out.

    The <ant> tasks need to be customized to your tests, depending on the kind of your tests you have to switch between core-test and gui-test. In our case the tests required the setting to be ui-test which means that the test will start an instance of eclipse.
    If the tests in your case work with the target set to "core-test" you can skip the part with the fake XServer and call the build.xml directly from CruiseControl. 
<target name="-runTestSuite" description="Should be called from allTests.xml only">
		
  <ant target="ui-test" antfile="${test.library.xml}" dir="${test.eclipse.host}">
    <property name="data-dir" value="${test.workspace} -clean" />
    <property name="plugin-name" value="org.codehaus.groovy.eclipse.tests" />
    <property name="classname" value="org.codehaus.groovy.eclipse.test.AllTests" />
    <property name="eclipse-home" value="${test.eclipse.host}" />
  </ant>
		
  <ant target="ui-test" antfile="${test.library.xml}" dir="${test.eclipse.host}">
    <property name="data-dir" value="${test.workspace} -clean" />
    <property name="plugin-name" value="org.codehaus.groovy.eclipse.core.test" />
    <property name="classname" value="org.codehaus.groovy.eclipse.core.JUnitTestSuite" />
    <property name="eclipse-home" value="${test.eclipse.host}" />
  </ant>
		
</target>
	
<target name="run" depends="-init,-runTestSuite" description="Run manually defined tests and create a report.">
  <delete dir="${test.result.dir}" />
  <mkdir dir="${test.result.dir}/xml" />

  <BundleFileLocator eclipseInstallationRoot="${test.eclipse.host}" bundleId="org.pluginbuilder.autotestsuite.application" filePath="JUNIT.XSL" property="junit.xsl.file" />
		
  <ant target="collect" antfile="${test.library.xml}" dir="${test.eclipse.host}">
    <property name="includes" value="*.xml" />
    <property name="output-file" value="${test.aggregated.report}" />
  </ant>
		
  <copy file="${test.eclipse.host}/${test.aggregated.report}" todir="${test.result.dir}/xml" />
  <style style="${junit.xsl.file}" basedir="${test.result.dir}/xml" destdir="${test.result.dir}/html" />
</target>
  • After you edited the run-tests.xml File to suit your needs you should try and run these tests locally. Make sure to have a zipped version of eclipse ready (with a corresponding entry in the pluginbuilder.config File). If you do have UI Tests you should see the new Eclipse Windows opening up. After the end of this Build you should be able to see the testresults in a html file that lies in the subdirectory results/testresults of your buildirectory.
  • If all these things are working on your computer, copy the configuration to your buildserver and change all the directory entries in the relevant files to match your server's environment. Have a look at the following Files: run-tests.properties, build_local.properties, pluginbuilder.config. Also make sure to have a zipped version of Eclipse (to run the tests in) as well as a normal installed Eclipse (used as BuildHost? for the plugins) ready on the buildserver. If the config files are adjusted you should be able to call the build.xml using ant on the buildserver. Check the testresults in the output folder again to make sure that all of the tests did run.
  • If you notice error messages in the testsections in the Ant log of your build you should read the next section. If they wasn't any errors and the testresults were good you can directly continue to the last section. If there were errors and if they were connected to the XServer you should read the next section.
  • Fake your Xserver: If your tests are UI-tests they expect an XServer to be available to run smoothly. Since having an actual X Server running on a (build)server would be rather weird we had to find another solution: Luckily there's one: Xvfb. A virtual framebuffer that offers us X without requiring any input or output hardware. Using Xfvb to set up a display and then having the UI tests running on this virtual display we were able to get all of the UI tests running.
    The Script we're using is based on Scripts that the people that brought Refactoring support into RDT 
    #!/bin/sh

#Simple script to get the UI tests of eclipse running
#using xvfb
#idea: mirko stocker (http://ifs.hsr.ch/)
#rkleeb, 20. 10. 2007

export TEST_DISPLAY=5

echo "Running tests on display $TEST_DISPLAY"

Xvfb :$TEST_DISPLAY -screen scrn 1280x1024x16  -br -fbdir /tmp -sp SecurityPolicy 2>&1 | grep -v "font" &


pid=`pgrep -n Xvfb`
sleep 1

export DISPLAY=:$TEST_DISPLAY

/var/lib/cruisecontrol/apache-ant-1.7.0/bin/ant -f build.xml cc-build

echo "Killing Xvfb with pid: $pid"
#important change since this buildserver is shared, only kill the Xvfb instance that has been started
#by a certain user
pkill -u rkleeb -n Xvfb
  • The last Step on this List would be to get all of this done automatically. In our case we used CruiseControl but I'm sure that alternatives like Continuum would also work out fine. If you made it all the way to this section the CruiseControl? Setup should be more or less a piece of a cake...Here's how our Project-Configuration (config.xml) looks like: 
    <project name="Groovy-Eclipse-With-Tests" buildafterfailed="false">

                <modificationset quietperiod="20">
                         <svn localworkingcopy="projects/groovy-refactoring/groovy-eclipse-with-tests/trunk" />
                </modificationset>

                <schedule interval="60">
                        <ant anthome="apache-ant-1.7.0"
                        antWorkingDir="projects/groovy-refactoring/groovy-eclipse-with-tests/"
                        buildfile="CallTestAndBuild.xml" target="build" />
                </schedule>

                <listeners>
                        <currentbuildstatuslistener file="logs/${project.name}/status.txt" />
                </listeners>

                <log dir="logs/${project.name}/">
                        <merge dir="/var/lib/cruisecontrol/projects/groovy-refactoring/groovy-eclipse-with-tests/build_dir/results/testresults/xml" />
                </log>
</project>
  • It's important to have your "merge dir" settings pointing to the directory where the Pluginbuilder writes his logs to, otherwise the testresults won't be shown in the CruiseControl web interface.
  • Usually you'd have your config calling an Ant script that checks out the Sources (I recommend [ PragAutomation ] out of the Pragmatic Starterkit if the whole CruiseControl subject should be new for you). But since the build.xml that has been generated already makes sure that the sources are checked out freshly every time we only have to call a tiny Ant script that calls our Shellscript that we introduced earlier: 
    <project name="cc-build" default="build">
        <target name="build">
                <exec executable="...[LONG DIR PATH].../runscript.sh">
                </exec>
        </target>
</project>

After a restart of Cruisecontrol your Plugin should be ready to be continuously built and tested.

  • If you should encounter problems:
    • The CruiseControl log files contain all the output that Ant generated during the build process.
    • Enable the verbose mode in Ant

Ressources