Xcode Cloud: Generating and translating TestFlight test notes automatically

Friday, June 9, 2023

During the Simplify distribution in Xcode and Xcode Cloud WWDC23 session Chris D’Angelo and Jason Wu from the Xcode team introduced a new Xcode Cloud feature that allows you to set TestFlight test notes for beta builds.

This is a long-awaited feature by many Xcode Cloud users including myself and it is going to help developers streamline their beta deployment processes even further with Xcode Cloud.

Setting test notes

As part of the TestFlight deployment process, Xcode Cloud now inspects the root of the project’s directory and looks for files with the name WhatToTest.<LOCALE>.txt inside a directory with the name TestFlight. For each of the files matching the pattern, Xcode Cloud uses the file’s contents to set the test notes for each locale.

It is important to note that you must provide a separate file for each locale you want to support and strictly follow the name convention. You can find a list of supported locales in Apple’s documentation.

I have spent some time adopting this new feature for my app QReate and I am going to show you how I have added autogenerated and auto-translated test notes for English (Great Britain and the United States), Spanish (Spain) and French (France) locales.

Creating the files

I first created a TestFlight directory at the root of my project and added a file for each English-speaking locale I wanted to support:

# Create the TestFlight directory
mkdir TestFlight
# Create a file for each locale
for locale in en-GB en-US; do
  echo "Bug fixes and improvements" > TestFlight/WhatToTest.$locale.txt
done

The terminal command above created the following directory structure, which matched the one expected by Xcode Cloud:

QReate
├── TestFlight
│   ├── WhatToTest.en-GB.txt
│   └── WhatToTest.en-US.txt

Running the workflow

Now that I had the first two files in place, I committed them to source control and then ran my TestFlight workflow, which uploads a new build to TestFlight for external testing, to verify everything worked as expected.

After the workflow finished executing, I checked App Store Connect and saw that the test notes had been set correctly for both locales:

Automatically generating test notes

While setting the test notes manually worked, I wanted to automate and streamline the process further by:

1. Automatically generating the test notes from the repository’s git log history.
2. Automatically translating the test notes to other languages. As I said at the beginning of the article, I wanted to also support Spanish and French test notes.

Luckily, Xcode Cloud has a feature called CI Scripts that allows you to run custom scripts and actions at different stages of the workflow.

I used this feature to create a ci_post_xcodebuild.sh script which is executed after the app is archived and generates and translates the test notes for all supported locales:

#!/bin/sh

if [[ $CI_WORKFLOW == "TestFlight" ]]; then
  pushd ..
    mkdir TestFlight
    pushd TestFlight
      for locale in en-GB en-US; do
        git fetch --deepen 3 && git log -3 --pretty=format:"%s" > WhatToTest.$locale.txt
      done
    popd
  popd
fi

Disclaimer: I used the logic described by Apple in their documentation to generate test notes automatically from the repository’s last three commits but I extended it and modified it to show how you can support multiple locales with minimal effort (as you will see in the following sections).

Automatic translation

I then used the AI-backed Google Translate API to translate the generated notes automatically into Spanish and French.

I created a small Swift command-line tool that takes in a String, a source and a target language and makes a network request to Google Translate API to get the translated notes:

import ArgumentParser
import Foundation

struct TranslateResponse: Codable {
    let data: DataModel
}

struct DataModel: Codable {
    let translations: [Translation]
}

struct Translation: Codable {
    let translatedText: String
}

@main
struct ReleaseNotesTranslator: AsyncParsableCommand {
    @Argument(help: "The text to be translated")
    var text: String

    @Argument(help: "The source language code")
    var source: String

    @Argument(help: "The target language code")
    var target: String


    func run() async throws {
        guard let apiKey = ProcessInfo.processInfo.environment["GOOGLE_API_KEY"] else {
            fatalError("Missing `GOOGLE_API_KEY` environment variable")
        }

        var urlComponents = URLComponents(string: "https://translation.googleapis.com/language/translate/v2")!
        urlComponents.queryItems = [
            .init(name: "q", value: text),
            .init(name: "source", value: source),
            .init(name: "target", value: target),
            .init(name: "format", value: "text"),
            .init(name: "key", value: apiKey)
        ]

        var request = URLRequest(url: urlComponents.url!)
        request.httpMethod = "POST"

        let (data, _) = try await URLSession.shared.data(for: request)
        let response = try JSONDecoder().decode(TranslateResponse.self, from: data)

        guard let translatedText = response.data.translations.first?.translatedText else {
            fatalError("Could not retrieve a translation")
        }

        print(translatedText)
    }
}

I then archived the code into a universal Apple binary (with x86_64 and arm64 architecture slices), committed it to source control and used it in my ci_post_xcodebuild.sh script to translate the originally generated test notes into Spanish and French:

#!/bin/sh

if [[ $CI_WORKFLOW == "TestFlight" ]]; then
  pushd ..
    mkdir TestFlight
    pushd TestFlight
      git fetch --deepen 3 && git log -3 --pretty=format:"%s" > WhatToTestSample.txt
      for locale in en-GB en-US; do
        cat WhatToTestSample.txt > WhatToTest.$locale.txt
      done

      # Translate the notes for each locale
      for locale in es-ES fr-FR; do
        language_code=$(echo $locale | cut -d- -f1)
        # Path to the compiled executable
        ./../release-notes-translator "$(cat WhatToTestSample.txt)" en $language_code > WhatToTest.$locale.txt
      done
      rm -rf WhatToTestSample.txt
    popd
  popd
fi

Note that the executable I made to translate release notes made use of an environment variable called GOOGLE_API_KEY, which I had to define as a secret in the Xcode Cloud workflow’s environment settings.

After running the workflow, I could see that the test notes had been generated and set correctly for all locales:

Compatibility

Xcode Cloud is deeply integrated with Xcode but it is not an Xcode feature. This means that you do not need Xcode to create, manage and run Xcode Cloud workflows. You can achieve the same result through the App Store Connect dashboard or by making network requests to the App Store Connect API.

What this means is that while these changes were announced during WWDC23, they do not require Xcode 15 to work. You can run your workflows from Xcode 14 with the changes described in this article and they will work in the same way as they would in Xcode 15! 🎉