In this tutorial, I will explain how to configure a continuous integration solution for your Android project, using Jenkins CI. In the meantime, I will show you how to run unit tests using Robolectric.

Before starting, please ensure you have the following tools:

  • Java SDK
  • Android SDK
  • Git (if you use the git integration)

History

  • 04/09/2015 - Corrects typos - Adds Product flavor trick - Adds custom test runner section
  • 03/31/2015 - Original publication

Install Jenkins

First, you need to download Jenkins CI. Visit official website and select the most appropriate version for your OS. Once installed, go to http://localhost:8080/ in your favorite web browser. You should see a running Jenkins server.

Well done! Let’s start by downloading some plugins. Navigate to Manage Jenkins > Manage Plugins. There, click on Available tab and search for the following plugins:

  • Android Lint
  • Git client
  • Github API (may be included in plugin below)
  • GitHub
  • Gradle
  • JaCoCo
  • JUnit (maybe already installed)

Start downloading and tick Restart once finished and no job running option.

Scaffold our application

Meanwhile plugins are downloaded, we are going to initialize our application. Launch Android Studio and create a new project with a single blank activity.

Then, initialize a new git repository in your current workspace. Add all files and commit them. Go to GitHub and create a new repository. Bind the remote URL to your local repository and push your new-born application.

Robolectric set-up

If you want to run unit tests in Android, this requires an emulator or a real device. Unfortunately, this operation may take 3-4 minutes before completing. That’s pretty bad, especially if you follow a TDD method.

Then came Robolectric. This tool allows developers to run unit tests without any emulator. Furthermore, it provides many tools for mocking up view components such as Activities or Fragments.

Ready? First, open your file explorer and go to your workspace. Navigate to app > src. Create a test folder in it. Then, recreate the same hierarchy than in your main folder. This hierarchy should match your package name, such as com.myCompany.myAppName.

.
├── androidTest
│   └── java
│       └── com
│           └── coshx
│               └── springbok
│                   └── ApplicationTest.java
├── main
│   ├── AndroidManifest.xml
│   ├── java
│   │   └── com
│   │       └── coshx
│   │           └── springbok
│   │               └── MainActivity.java
│   └── res
└── test
    └── java
        └── com
            └── coshx
                └── springbok

Done? Fantastic. We created a test folder, where all our unit tests will be stored. Indeed, the default folder (androidTest) stores only integration tests. If you put some UTs in it, you will not be able to run them without emulator.

Go back to Android Studio. If you refresh your project (File > Synchronize), you should see nothing new. Hold on mate, we are close.

Open the build.gradle file of your project (not your module’s one, should be at the root of your workspace). Add this node in dependencies one:

// Robolectric
classpath 'org.robolectric:robolectric-gradle-plugin:1.0.1'

Now, open the build.gradle file of your module (should be in appfolder). Edit it to match the code below:

apply plugin: 'com.android.application'
apply plugin: 'org.robolectric' // Enables Robolectric support

android {
    compileSdkVersion 21
    buildToolsVersion "21.1.2"

    defaultConfig {
        applicationId "com.coshx.springbok"
        minSdkVersion 15
        targetSdkVersion 21
        versionCode 1
        versionName "1.0"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }

    productFlavors {
        unitTest // Creates a new scope which wraps only unit tests
    }

    sourceSets {
        unitTest {
            java {
                srcDir 'src/test/java' // New scope includes our unit test folder
            }
        }
    }

    // Prevent conflicts between Robolectric's dependencies
    packagingOptions {
        exclude 'META-INF/DEPENDENCIES.txt'
        exclude 'META-INF/LICENSE.txt'
        exclude 'META-INF/NOTICE.txt'
        exclude 'META-INF/NOTICE'
        exclude 'META-INF/LICENSE'
        exclude 'META-INF/DEPENDENCIES'
        exclude 'META-INF/notice.txt'
        exclude 'META-INF/license.txt'
        exclude 'META-INF/dependencies.txt'
        exclude 'META-INF/LGPL2.1'
        exclude 'LICENSE.txt'
        exclude 'LICENSE'
    }
}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    compile 'com.android.support:appcompat-v7:21.0.3'

    // Espresso
    androidTestCompile 'com.android.support.test.espresso:espresso-core:2.0'
    androidTestCompile 'com.android.support.test:testing-support-lib:0.1'

    // Unit testing dependencies
    unitTestCompile('junit:junit:4.12') { // Prevent duplication conflicts
        exclude module: 'hamcrest-core'
        exclude module: 'hamcrest-library'
        exclude module: 'hamcrest-integration'
    }
    unitTestCompile 'org.hamcrest:hamcrest-core:1.1'
    unitTestCompile 'org.hamcrest:hamcrest-library:1.1'
    unitTestCompile 'org.hamcrest:hamcrest-integration:1.1'

    unitTestCompile('org.robolectric:robolectric:2.4') {
        exclude module: 'classworlds'
        exclude module: 'commons-logging'
        exclude module: 'httpclient'
        exclude module: 'maven-artifact'
        exclude module: 'maven-artifact-manager'
        exclude module: 'maven-error-diagnostics'
        exclude module: 'maven-model'
        exclude module: 'maven-project'
        exclude module: 'maven-settings'
        exclude module: 'plexus-container-default'
        exclude module: 'plexus-interpolation'
        exclude module: 'plexus-utils'
        exclude module: 'wagon-file'
        exclude module: 'wagon-http-lightweight'
        exclude module: 'wagon-provider-api'
    }
}

// Robolectric config
robolectric {
    // Configure includes / excludes
    include '**/*Test.class'
    exclude '**/espresso/**/*.class'

    // Configure max heap size of the test JVM
    maxHeapSize = '2048m'

    // Configure the test JVM arguments - Does not apply to Java 8
    jvmArgs '-XX:MaxPermSize=512m', '-XX:-UseSplitVerifier'

    // Specify max number of processes (default is 1)
    maxParallelForks = 4

    // Specify max number of test classes to execute in a test process
    // before restarting the process (default is unlimited)
    forkEvery = 150

    // configure whether failing tests should fail the build
    ignoreFailures false

    // use afterTest to listen to the test execution results
    afterTest { descriptor, result ->
        println "Executing test for ${descriptor.name} with result: ${result.resultType}"
    }
}
//end Robolectric config

Did you notice? I used both androidTestCompile and unitTestCompile. The first one specifies a dependency for integration tests, where an emulator is needed (default behaviour). Second one specifies unit testing dependencies.

Synchronize your project and ensure you have no error at this point. If you have any troubles with Proguard (such as a Please resolve the warnings first error), open the proguard-rules.pro file at the root of your project. Then, append those lines:

-dontwarn **
-dontnote **

If you have never heard about Proguard, it is the tool for streamlining your build. It inspects all your code and removes parts of it that it considers as useless. Using Proguard may damage your project if you do not know how to use it correctly. For disabling this guy, please be sure minifyEnabled is set to falsein your buildTypes node (from your build.gradle).

Let’s create our first test. Firstly, build a simple bean class in your application:

public class User {
    private String _name;

    public String getName() {
        return _name;
    }

    public void setName(String value) {
        _name = value;
    }
}

Awesome. Now create the associated test in the empty unit test folder:

@Config(emulateSdk = 18, reportSdk = 18)
@RunWith(RobolectricTestRunner.class)
public class UserTest {

    @Test
    public void UserNameTest() {
        User u = new User();

        u.setName("foo");

        Assert.assertEquals("foo", u.getName());
    }
}

Imports should be automatic. Otherwise, check your dependencies again, in your Gradle build file.

Ready? Let’s test if everything is fine! In your terminal, run this command (in your workspace):

./gradlew clean test

Once done, you should see something close to this:

:app:testUnitTestRelease
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier; support was removed in 8.0
objc[58353]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Executing test for UserNameTest with result: SUCCESS
:app:test

BUILD SUCCESSFUL

Total time: 2 mins 44.831 secs

Amazing! You did it! If you want to be sure your build will fail if any test if failing, add a failing test to your class:

@Test
public void failingTest() {
    Assert.assertFalse(true);
}

Then, run ./gradlew test again. You should have a build failure notification.

Every test class you will put in the src/test folder can be run as a unit test, using previous shell command.

Android linter

For display Android linting result in Jenkins, we need to enable this feature in our project’s config. For that, open your Gradle module file again and paste this in the android node:

lintOptions {
    // I prefer avoiding a build fail if there is any warning, for they are only warnings
    abortOnError false
}

If you want to display a warning for testing purpose, edit your activity_main layout and replace the Hello World string reference by a real string. You should see a warning in the editor’s gutter (yellow mark).

Compute code coverage using JaCoCo

The last configuration step is to enable code coverage. We are going to use JaCoCo for this.

Open again your build.gradle. Change the buildTypes node by the following one:

buildTypes {
    debug {
        debuggable true
        minifyEnabled false
        proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        testCoverageEnabled true
    }
    release {
        minifyEnabled false
        proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        testCoverageEnabled false
    }
}

Then, paste this configuration at the of the file:

// Jacoco config
apply plugin: 'jacoco'

jacoco {
    // Use this version for upper ones are broken (with Gradle)
    // https://github.com/jacoco/jacoco/issues/288
    toolVersion = "0.7.1.201405082137"
}

// Edit covered scope if needed
// For my part I like having the coverage of both application and tests
def coverageSourceDirs = [
        '../app/src'
]

task jacocoTestReport(type: JacocoReport, dependsOn: "test") {
    group = "Reporting"

    description = "Generate Jacoco coverage reports"

    classDirectories = fileTree(
            dir: '../app/build/intermediates/classes',
            excludes: ['**/R.class',
                       '**/R$*.class',
                       '**/*$ViewInjector*.*',
                       '**/BuildConfig.*',
                       '**/Manifest*.*']
    )

    additionalSourceDirs = files(coverageSourceDirs)
    sourceDirectories = files(coverageSourceDirs)
    executionData = files('../app/build/jacoco/testUnitTestDebug.exec')

    reports {
        xml.enabled = true
        html.enabled = true
    }

}
//end Jacoco config

This code adds a new task, jacocoTestReport. Running it will create XML and HTML reports which show your current coverage. You can customize the scope, using the coverageSourceDirs var. Also, running jacocoTestReport task will run test before.

To test it, run this command in your terminal:

./gradlew clean jacocoTestReport

You should have a success notification. Then, navigate to app > build > reports > jacoco > jacocoTestReport. You should find both XML and HTML reports. Glance at the HTML one and see you current coverage. Your User class should be fully covered.

Wonderful! You’re done with Android configuration. You can commit your changes to your repo.

Create a Jenkins job

Now, let’s move back to Jenkins. Firstly, be sure that plugins have been installed. Otherwise, wait a little bit more.

Global configuration

Go to Manage Jenkins > Configure system. Then, follow these steps:

  1. Global properties tab: Add a new key-value pair. Key is ANDROID_HOME and value is the path to your local Android SDK.
  2. JDK installations: Set a name for your JDK and fill the path value (JAVA_HOME).
  3. Git: I use the Install automatically option. If you prefer using a custom version, feel free to change it.
  4. Gradle: Ditto than Git. I use an automatic installation and 2.3 version. If you want to use only your local wrapper (gradlew file), you can skip that step.
  5. GitHub web hook: I chose the Let Jenkins auto-manage hook URLS option. Using it, Jenkins automatically hooks up with Github and let it notify time a modification is done. Besides, thanks to that plugin, we do not need to provide our credentials as plain text. Instead, we are going to generate an authentication token. If you do not know how to process, go to GitHub and navigate to your account settings. Open Applications tab. There, you can generate tokens in Personal access tokens section. Generate a new one using the name you wish and keep default authorizations (repo, public_repo, gist and user). Finally, go back to Jenkins and provide your fresh token and your username.

Voilà! Do not forget to save your settings.

Settings

On the Jenkins’ homepage, click on New Item. Fill out a name and flag your project as a Freestyle one.

Once submitted, you can edit the configuration of your new project. Do the following operations:

  1. Main tab: Enter the address of your repository (GitHub project field). This provides only a shortcut from your dashboard.
  2. Source code management: Select the Git option. Enters the URL of your repo again and your credentials. Then, customize branches to build, if you wish (by default, master only).
  3. Build triggers: Check Build when a change is pushed to GitHub option.
  4. Build section: Add new build step - Invoke Gradle script. Select Use Gradle Wrapper option for using local wrapper or select an installed version (defined in Jenkins’ main configuration, see above). Add following tasks: clean build. Finally, check Force GRADLE_USER_HOME to use workspace option.

Now, we are going to add different post-build actions. Using those, we will publish different reports (coverage, unit testing results…) at the end of the build.

  1. Publish Android Lint results: Fill out Lint files field with **/lint-results.xml.
  2. Archive the artifacts: At the end of each build, we will archive apk files. Fill input with **/*.apk.
  3. Publish JUnit test result report: Enters **/build/test-results/**/*.xml for Test report XMLs field.
  4. Record JaCoCo coverage report: Enters values below. They should match values we used in our build.gradle.
    • Path to exec files: **/build/jacoco/*.exec
    • Path to class directories: **/build/intermediates/classes
    • Path to source directories: **/src
    • Exclusions: **/R.class, **/R$*.class, **/*$ViewInjector*.*, **/BuildConfig.*, **/Manifest*.*
undefined
Figure: Project configuration


Here we go! Save your new project :)

Now it’s time for testing! Click on build now and go to console output once build has started. You should be able to follow a trace of your build.

Unfortunately report charts are not available with a single build. You need to build your project again. At the end, you should be able to review a similar dashboard:

undefined
Figure: Project dashboard


Finally, if you want to test GitHub integration, you need to deploy your server somewhere. Once done, in your repo’s settings, check your server is hooked up to the repository in Webhooks & Services section. If you push new changes, a build should automatically start. You can also trigger a manual build using Test service feature in GitHub.

Oh my babe, just say goodbye

That’s insane.

You have just deployed your first Jenkins server with a full Android integration support. You know how to run unit tests without crappy emulator anymore. You can review your code coverage. You have periodic releases.

You’re so awesome!

Hope you’ve appreciated this article. You can find my test application here. If you have any questions, please post a comment, tweet or email me a lovely message ([email protected]).

Go further

Change build status according to thresholds

Depending on your criterions, you may be interested in setting your own conditions for raising an error. In your project’s configuration, Jenkins allows you to do whatever you wish. Then, you can flag a build as failed if your code is not covered enough. Also, if there is any lint warning, you can trigger a failure. Ditto for unit tests.

Custom build name

By default, build names are really short (only a number). If you have a repository with multiple branches you can feel lost pretty fast.

You can change that by installing the Build Name Setter plugin. Customize your build name in your project’s configuration.

For my part, I use the following one:

#${BUILD_NUMBER}-${GIT_BRANCH}-${GIT_REVISION}

More plugins

In this article, I helped you setting up the minimum requirements an integration solution should match. You can find extra plugins below:

  • Checkstyle: Ensures committed code matches team’s styleguides
  • DRY: Analyses your code and spots if some parts are similar to each other
  • FindBugs: Analyses your bytecode code and finds potential glitches
  • PMD: Analyses your code and finds potential glitches
  • Warnings: Displays compiler warnings
  • Task Scanner - Spots FIXME and TODO annotations within your code

And finally, an extra:

Static Code Analysis: Gathers all the results from plugins above and displays them in a same graph.

Product flavor trick

If you have followed my tutorial and have deployed your application, you may have noticed the build is quite big. Unfortunately, when building the application, Gradle is including all the unit tests and relative dependencies. Your application does not need them for running.

Here is a small trick for streamlining your build. Open again your module’s build.gradle. Within it, add a new product flavor:

productFlavors {
    app
    unitTest
}

app is the default flavor. Once synchronized, you can switch from a flavor to another using Build Variant panel in Android Studio (in the bottom right corner by default). If you select either appDebug or appRelease, you should see your unit test folder anymore. That’s what we aimed. Now, if you deploy, your build is going to be really lighter. Move back to unitTest variant for being able to edit your unit tests again.

Introducing this flavor will also change your Gradle tasks. If you want to run your unit tests, you need to run gradle assemble testUnitTestDebug from now on, instead of test only.

Define a test runner

Well, if you try to use the basic tools from Robolectric, such as mocking activities, and run your tests, it may crash. This problem is due to missing resources Robolectric needs to have for mocking activities such as resources or Manifest.

For automatically importing these resources, we need to define a custom test runner.

Creates a new class in your unitTest folder:

public class GradleRobolectricTestRunner extends RobolectricTestRunner {
    private static final String PROJECT_DIR =
        getProjectDirectory();

    // Include manifest
    private static final String MANIFEST_PROPERTY =
        PROJECT_DIR + "src/main/AndroidManifest.xml";

    // Include compiled resources (not the raw ones)
    private static final String RES_PROPERTY =
        PROJECT_DIR + "build/intermediates/res/app/debug/";

    // Define SDK to use
    private static final int MAX_SDK_SUPPORTED_BY_ROBOLECTRIC =
        Build.VERSION_CODES.JELLY_BEAN_MR2;

    public GradleRobolectricTestRunner(final Class<?> testClass) throws Exception {
        super(testClass);
    }

    private static String getProjectDirectory() {
        String path = "";
        try {
            File file = new File("..");
            path = file.getCanonicalPath();
            path = path + "/app/";
        } catch (IOException ex) {
        }
        return path;
    }

    @Override
    public AndroidManifest getAppManifest(Config config) {
        return new AndroidManifest(
            Fs.fileFromPath(MANIFEST_PROPERTY),
            Fs.fileFromPath(RES_PROPERTY)
        ) {
            @Override
            public int getTargetSdkVersion() {
                return MAX_SDK_SUPPORTED_BY_ROBOLECTRIC;
            }

            @Override
            public String getThemeRef(Class<? extends Activity> activityClass) {
                return "@style/AppTheme"; // Needs to define default theme
            }

        };
    }
}

Perfect. You can mock activities and use any tool from Robolectric:

// Notice you do not need the @Config annotations anymore
// Every setting is defined in this custom test runner
@RunWith(GradleRobolectricTestRunner.class)
public class MainActivityTest {
    private MainActivity _activity;

    @Before
    public void setUp() {
        _activity = Robolectric.setupActivity(MainActivity.class);
    }

    @Test
    public void notNull() {
        Assert.assertNotNull(_activity);
    }
}

Before running your tests, be sure you run Gradle assemble task. This command will, in particular, generate compiled resources, needed for unit tests.

If you still have errors, please be sure you are not trying to mock an ActionBarActivity. Unfortunately, at that point, Robolectric cannot mock this kind of activities.

Mac hack

If you run Jenkins on a Mac, as I do, you may be pretty annoyed for stopping Jenkins’ daemon. Here is a little trick I created, based on the official documentation:

# Jenkins
jenkins() {
    start() { sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist; }
    stop() { sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist; }
    if [ "$1" = "restart" ]; then
        stop
        start
    elif [ "$1" = "start" ]; then
        start
    elif [ "$1" = "stop" ]; then
        stop
    else
        echo "Unsupported command"
    fi
}
alias jenkins=jenkins

Paste this code at the bottom of your bash_profile. Now, you can run these commands: restart, start and stop.