Xcode11 Code Coverage with Fastlane, Slather, and Pipelines

How to publish Xcode 11 code coverage results into Azure DevOps pipelines using fastlane and slather

Maybe some of you are currently trying to generate code coverage results from your unit tests. In this case, “generate” means running your unit tests in Xcode, waiting for the results and showing them in a nice report, maybe somewhere on a CI machine (e.g., Azure DevOps, in our case).

The struggle:

The problem that we are facing is that, after running our tests, how do we publish them on our Azure DevOps machine?

This may get us in some trouble regarding how we can use Azure DevOps and our code coverage results from Xcode in order to create a nice-looking test report. Do we need to upload the coverage files somewhere? Do we have to regenerate the code coverage inside a Terminal or Command Line in Azure DevOps?

The solution:

The fastest solution in this situation is to use fastlane’s slather tool.

First, a few words about fastlane, in order to get you familiar with it:

Fastlane lets you simply create “lanes” (e.g., methods/functions) in order to configure your app for either testing, creating builds, releases, etc. All the actions are done inside a plain and simple text file.

Fastlane drastically reduces the time you spend on building and testing your applications. You just need to create some lanes, add some parameters and you’re done. Then you just need to “call” fastlane to perform its magic and everything is solved. “Calling” fastlane simply means using a Terminal and running your lanes, like this:

fastlane ‘LaneName’

What is slather and how to install it?

Slather is a framework that lets you “Generate test coverage reports for Xcode projects & hook it into CI.”

Installation: Simply use the Terminal again and run the following command:

gem install slather

And with that, installation is complete. From this point on, you can simply create yourself a lane and use the slather framework inside. Also, you need to make sure that, before using slather, you run your unit tests and gather the code coverage.

*Note: Use slather in the same folder you ran your tests.

Below is an example of how to create a fastlane lane and use it to run your unit tests.

scan(scheme: "SchemeName",
         code_coverage: true,
clean: true,
workspace: "./MyApp.xcworkspace",
derived_data_path: "../testOutput/",
skip_build: true)

* skip_build – lets you scan your tests faster, without the need to previously build your project

After running your unit tests using scan, use slather as described below:

slather(build_directory: "../testOutput/",
   scheme: "SchemeName",
   workspace: "./MyApp.xcworkspace",
   proj: "./MyApp.xcodeproj",
   output_directory: "../testOutput/slatherOutput", 
   cobertura_xml: true,
   ignore: ["Pods/*", "ThirdParty/*", “Frameworks/*"])

Azure DevOps

In order to use slather for our Azure DevOps pipeline, we need to gather an XML file from slather. That’s why the cobertura_xml parameter is set to true.

This will generate an XML file called cobertura.xml which will contain nicely grouped info about our code coverage.

The ignore parameter helps a lot: We do not want to see code coverage results related to Pods or other third parties because that is not of interest to us. Specifying paths inside the ignore parameter will remove those coverage results from our xml file.

* Pods/* path means that slather will ignore the files from the folder Pods and all its subfolders and files when generating the Cobertura xml file.

The Publish Code Coverage task requires the usage of either JaCoCo or Cobertura plugin in order to publish your results. We will use Cobertura for our reports.

For the plugin to work, we need an XML file that contains our code coverage results and so, the Azure DevOps task will publish our results directly in the pipeline page, in a tab called “Code Coverage.”

We will use the cobertura.xml generated earlier. In order for the “Publish Code Coverage” task to work, we need to specify the path for our generated XML file in the “Summary” section of the task. We will choose the path that contains our xml file.

In addition, even if it is not mandatory, we can specify the “Sources directory” path. In this way, we will inform the Publish Code Coverage task of where our source code is.

After everything is set up, simply press the Queue button and wait for it to complete. Then the “Code Coverage” tab will show up (after the pipeline has finished all its tasks) and will provide you with a view about your code coverage results, like below:

Below this view, you can also see the line coverage for each of your files inside the project and some nice percentages which, of course, can always be improved.

Final thoughts & recommendations:

As a Software Engineer in the iOS community, I researched other tools like fastlane’s xcov or Apple’s xccov and found out that it can take so much time to find the proper coverage files and then use them accordingly to upload the results to Azure’s DevOps pipeline. 

Start your work using slather, as it is very simple to use along with fastlane!

I strongly suggest considering the approach presented here as I researched using the above-mentioned tools and found that fastlane’s slather is an improved and faster way of generating your code coverage. As previously stated, slather only needs a bit of parameter configuration and then you can safely use it with any other problems.

My recommendation?  Work more with fastlane and pipelines, as it is a nice way to improve your skills regarding CI and all the processes involved in packaging and distributing your app to TestFlight or AppStore.