How to programmatically parse the contents of an XCResult bundle
No one is immune from shipping critical bugs to production, but Runway helps you limit the amount of havoc that can cause.
An XCResult bundle is a package or directory that contains detailed information about the results of running a set of tests. These bundles are generated by Xcode (or by xcodebuild in the command line) and provide a wealth of information about the tests that were run, including the test’s name, duration, status, and any attachments generated by them such as screenshots or logs.
In Xcode, you can find and inspect the XCResult bundle after a test run by going to the ‘Report Navigator’ and selecting the bundle you are interested in from the list:
If you’d like to share the bundle with someone else, you can right-click on it from the ‘Report Navigator’ and select ‘Show in Finder’ to open the directory where the bundle is located. All .xcresult bundles are generated in your app’s Logs/Test directory in Derived Data whether you run the tests with xcodebuild from the command line or in Xcode, and you can just double-click on the .xcresult file to open it in Xcode and inspect the bundles’ contents.
Parsing an XCResult bundle
When you run an app’s tests in a CI/CD environment, XCResult bundles become even more important as, without them, the only information you would have about test failures would be the logs of the xcodebuild command. Furthermore, access to the CI/CD machines is usually restricted and cumbersome, so retrieving the .xcresult bundle for a specific run is not always straightforward.
This is why it is usually a good idea to have your CI/CD service of choice upload the XCResult bundle as an artifact to your workflows on failing test runs so that developers can download it and inspect the results. While this is a major improvement in developer experience, the feedback is not instant, as it requires developers to download the bundle and open it on their machine.
Wouldn’t it be great if you could programmatically parse the contents of an XCResult bundle and extract the information you need without having to open Xcode instead? This way, you could automate the process of inspecting test results and provide instant feedback to developers about test failures. This sounds great in principle, but when you inspect the contents of an .xcresult bundle, you soon realize that the contents are not human-readable, which makes the task of parsing them programmatically a bit more challenging:
Parsing the bundle’s contents
Thankfully for us, there are tools out there that make our lives easier when it comes to parsing the contents of an XCResult bundle. One such library, which happens to be written in Swift and that we will use for the rest of this article, is XCResultKit by David House.
Let’s consider that we have an app with two test bundles, one for unit tests and another one for UI tests. We run the tests and they fail. Upon inspection of the .xcresult bundle, we find that the unit tests are all passing but we have one UI test that is failing:
Over the next sections, we will learn how to extract information from this test such as the screen recording of the test’s failure.
Initialising the library
To get started, we first need to import the library into our project as a Swift Package. In this case, we will build a Swift executable that will use XCResultKit to extract information from an .xcresult bundle:
// swift-tools-version: 6.0
import PackageDescription
let package = Package(
name: "ResultAnalyzer",
platforms: [
.macOS(.v13)
],
dependencies: [
.package(url: "https://github.com/apple/swift-argument-parser.git", exact: "1.5.0"),
.package(url: "https://github.com/davidahouse/XCResultKit.git", exact: "1.2.0")
],
targets: [
.executableTarget(
name: "ResultAnalyzer",
dependencies: [
.product(name: "ArgumentParser", package: "swift-argument-parser"),
.product(name: "XCResultKit", package: "XCResultKit")
]
),
]
)In the main file of our executable, we can now import the library, ask for a path to a .xcresult bundle and initialise an XCResult object with the path provided by the user:
import ArgumentParser
import Foundation
import XCResultKit
@main
struct XCResultAnalyzer: ParsableCommand {
@Argument(help: "The path to an `.xcresult` bundle")
var bundle: String
func run() throws {
guard let url = URL(string: bundle) else { return }
let result = XCResultFile(url: url)
}
}Getting the invocation record
The first step to reading content from the bundle is to get the information record. This record contains all metadata and information to retrieve the rest of the data from the bundle:
func run() throws {
guard let url = URL(string: bundle) else { return }
let result = XCResultFile(url: url)
guard let invocationRecord = result.getInvocationRecord() else { return }
}The information record contains some top-level information about the test run, such as the actions that took place, a detailed summary of the issues encountered and metrics from the test run:
func run() throws {
guard let url = URL(string: bundle) else { return }
let result = XCResultFile(url: url)
guard let invocationRecord = result.getInvocationRecord() else { return }
print("✅ Ran \(invocationRecord.metrics.testsCount ?? .zero) tests and skipped \(invocationRecord.metrics.testsSkippedCount ?? .zero)")
print("❌ \(invocationRecord.issues.testFailureSummaries.count) test failures")
print("🧐 Ran actions: \(invocationRecord.actions.compactMap { $0.testPlanName })")
}Running the executable with the same .xcresult bundle we inspected earlier, we get the following output:
✅ Ran 3 tests and skipped 0
❌ 1 test failures
🧐 Ran actions: ["AutomatedTesting"]Getting information about a test
Getting specific information about a given test is a bit more involved, as you need to iterate through all the actions in the bundle, get the test plan information and only then you can access specific information about individual tests.
Let’s start by retrieving all failing tests from the bundle:
func run() throws {
guard let url = URL(string: bundle) else { return }
let result = XCResultFile(url: url)
guard let invocationRecord = result.getInvocationRecord() else { return }
// 1
let testBundles = invocationRecord
.actions
.compactMap { action -> ActionTestPlanRunSummaries? in
guard let id = action.actionResult.testsRef?.id, let summaries = result.getTestPlanRunSummaries(id: id) else {
return nil
}
return summaries
}
.flatMap(\.summaries)
.flatMap(\.testableSummaries)
let allFailingTests = testBundles
// 2
.flatMap(\.tests)
// 3
.flatMap(\.subtests)
.filter { $0.testStatus.lowercased() == "failure" }
}Let’s look back at the bundle and map its structure to the comments in the code:
Now that we have the failing tests, we can get the summary with all their steps, retrieve the screen recording attachment from the first step and export it:
func run() throws {
// ...
let screenRecordings = allFailingTests
.compactMap { test -> ActionTestSummary? in
guard let id = test.summaryRef?.id else { return nil }
return result.getActionTestSummary(id: id)
}
// 1
.flatMap(\.activitySummaries)
// 2
.first?
// 3
.attachments
.filter { $0.name == "kXCTAttachmentScreenRecording" && $0.uniformTypeIdentifier == "public.mpeg-4" } ?? []
for screenRecording in screenRecordings {
let tempFileDirectory = URL.temporaryDirectory
result.exportAttachment(attachment: screenRecording, outputPath: tempFileDirectory.path())
}
}Let’s look at the bundle again and map its structure to the comments in the code:
And that’s it! Next time you run the executable with the path to the .xcresult bundle, you will get the screen recording of the failing test exported to your temporary directory, ready to be shared wherever you need it.