Coming in Swift 5.9: Network requests in Swift package plugins

Wednesday, February 22, 2023

The merge of PR #6114 in the Swift Package Manager repository confirms one of the features I have long been waiting for: the ability to make network requests in Swift package plugins. This feature will be officially available from Swift 5.9 onwards, but it is already available on the latest development snapshots of Swift and ships with Xcode 15 and onwards 🎉.

The moment I found out that this feature had landed in the repository’s main branch, I knew I had to try it out and write about it in the same way I did when extensible build tools were first introduced.

An overview

As of the release of Swift 5.9, a new type of permission will be added to Swift package command plugins: allowNetworkConnections. This permission will allow a plugin to make network requests to a Docker daemon, a local or remote server or to Unix domain sockets.

The initial PR also adds the option to specify a list of ports within the scope that the plugin is allowed to make requests to. At the time of writing this article though, there is no way to provide a list of domains that can be requested from the plugin’s code when allowing remote network requests.

When specifying that a plugin requires the allowNetworkConnections permission, we must also specify a scope which determines the requests that can or cannot be made from the plugin’s code.

The available scopes at the moment are:

• docker - allows the plugin to communicate with a Docker daemon.
• local - allows the plugin to communicate with a server local to the system where the plugin is executing. This enum case has an associated value ports, to only allow requests to local servers running on any of the ports specified.
• all - allows the plugin to make a request to any domain, both local and remote. This enum case has an associated value ports, to only allow requests to servers (local or remote) running on any of the ports specified.
• unixDomainSocket - allows the plugin to communicate with a Unix domain socket.

In the next few sections, I will go through a few examples of how the new permission can be set and what the implications of setting each scope are in practice.

Communicating with docker

Let’s consider an example of a Swift package plugin that communicates with Docker to build the package’s product. The plugin will be able to start a docker container and execute swift build on it. In this case, it will make use of one of Swift’s official main nightly build containers to compile a Swift executable target into a amazonlinux2 compatible binary.

Note that the container I used for this example has version 5.9 of Swift package manager as it is a development snapshot built overnight from Swift’s main branch. This example wouldn’t work with the latest stable docker image of Swift, as it wouldn’t have the allowNetworkConnections permission available.

First, let’s define a plugin target in the Package.swift file and specify the swift-tools-version to be 5.9:

// swift-tools-version: 5.9
import PackageDescription

let package = Package(
    name: "PluginsPermissionsTest",
    products: [
        .executable(
            name: "PluginsPermissionsTest",
            targets: ["PluginsPermissionsTest"]
        )
    ],
    dependencies: [
    ],
    targets: [
        .target(
            name: "PluginsPermissionsTest",
            dependencies: []
        ),
        .plugin(
            name: "Docker",
            capability: .command (
                intent: .custom(verb: "compile-to-amazonlinux2", description: "Compile executable to amazon linux 2"),
                permissions: [
                    .allowNetworkConnections(
                        scope: .docker,
                        reason: "The plugin must connect to the docker daemon to compile Swift code to amazonlinux2"
                    )
                ]
            )
        )
    ]
)

Then, let’s write the plugin’s implementation:

@main
struct Docker: CommandPlugin {
    func performCommand(context: PluginContext, arguments: [String]) throws {
        // 1
        let docker = try context.tool(named: "docker").path

        // 2
        try shell(command: docker, arguments: [
            "run",
            "--rm",
            "--volume", "\(context.package.directory.string):/src",
            "--workdir", "/src",
            "swiftlang/swift:nightly-amazonlinux2",
            "swift", "build", "-c", "release"
        ])
    }
}

Let’s go through the plugin implementation above in more detail:

1. Retrieve the path to the docker executable from the context. In this case, the context will know to use the system’s docker executable.
2. Run a command using a helper method shell, which wraps a call to the Process API, executes the command with the given arguments and returns a String with the command’s output. I won’t go into too much detail about what the command itself does, but all you need to know is that it will run swift build on a swiftlang/swift:nightly-amazonlinux2 docker image. The output of this command will produce a linux-compatible executable of the PluginsPermissionsTest product in the current working directory.

We can then run the plugin from the command line like so:

# Set the version of Swift to a development snapshot with Swift 5.9
swiftenv local DEVELOPMENT-SNAPSHOT-2023-02-19-a

# Run the plugin
swift package compile-to-amazonlinux2 --allow-network-connections docker

If you want to learn more about how to download and install development snapshots of Swift, check out my article on the topic.

Before the plugin executes, it will ask for permissions to make network requests to Docker. As shown in the bash code snippet above, you can also make the plugin command invocation not interactive by passing it the --allow-network-connections flag. This is very useful for running the plugin in a CI environment or when you have to call it repeatedly.

Once the plugin has all necessary permissions, it will interact with Docker’s daemon and will output a file under: ‘.build/aarch64-unknown-linux-gnu/release/PluginsPermissionsTest’:

Communicating with a local server

Let’s now see how a plugin can be used to communicate with a local server. Again, the first thing to do is to define a plugin target and this time give it an allowNetworkConnections permission with local scope so that it can fetch the contents of config.json from a local server running at port 30 and a writeToPackageDirectory permission so it can write such contents to a file:

// swift-tools-version: 5.9
import PackageDescription

let package = Package(
    name: "PluginsPermissionsTest",
    // ...
    targets: [
        .plugin(
            name: "FetchConfig",
            capability: .command (
                intent: .custom(verb: "fetch-config", description: "Fetch a `config.json` resource and embed it in the package"),
                permissions: [
                    .allowNetworkConnections(
                        scope: .local(ports: [30]),
                        reason: "The plugin must connect to a local server running at port 30 to fetch a resource"
                    ),
                    .writeToPackageDirectory(reason: "Write the fetched `config.json` to the package's root directory")
                ]
            )
        )
    ]
)

Let’s now write the code for a plugin which fetches a configuration file (.json) from a local server and drops it in the package’s root directory:

@main
struct FetchConfig: CommandPlugin {
    func performCommand(context: PluginContext, arguments: [String]) throws {
        // 1
        let curl = try context.tool(named: "curl").path

        // 2
        let response = try shell(executable: curl, arguments: [
            "http://localhost:30/config.json",
            "-o",
            context.package.directory.appending(subpath: "config.json").string
        ])

        // 3
        print(response)
    }
}

Let’s go through the plugin’s implementation above in more detail:

1. Retrieve the path to the curl executable from the context. In this case, the context will know to use the system’s curl executable.
2. Run a command using a helper method shell, which wraps a call to Process, executes the command with the given arguments and returns a String with the command’s output. The command executes a network request to http://localhost:30/config.json and saves the response to a config.json file at the package’s root directory using curl.
3. Print the command’s output to the terminal.

To test this, we can create a dummy config.json file and run a local server using python3 in the directory where the new file is located:

# Create a dummy config.json file
echo '{"foo": "bar"}' > ~/Desktop/config.json

# Run a local server on port 30
python3 -m http.server 30

Now that the local server is running, we can run the plugin from the command line like so:

# Set the version of Swift to a development snapshot with Swift 5.9
swiftenv local DEVELOPMENT-SNAPSHOT-2023-02-19-a

# Run the plugin
swift package fetch-config                  \
    --allow-writing-to-package-directory    \
    --allow-network-connections local

We should now see curl’s output in the terminal and a new file (config.json) at the root of the package’s directory:

Communicating with a remote server

Let’s now modify the FetchConfig plugin we have just created to query a remote endpoint instead:

@main
struct FetchConfig: CommandPlugin {
    func performCommand(context: PluginContext, arguments: [String]) throws {
        let curl = try context.tool(named: "curl").path

        let response = try shell(executable: curl, arguments: [
            "https://httpbin.org/json",
            "-s",
            "-o",
            context.package.directory.appending(subpath: "config.json").string
        ])

        print(response)
    }
}

Note that for testing sake I am using httpbin.org to get a JSON response which I can then write to a file. You can use any remote endpoint you want.

If we run the plugin again now, we will get the following error:

This error occurs because the plugin is trying to make a network request to a remote endpoint, and we have not yet granted it permission to do so. Let’s fix that by changing the scope of the allowNetworkConnections permission from local to all in the Package.swift file:

// swift-tools-version: 5.9
import PackageDescription

let package = Package(
    name: "PluginsPermissionsTest",
    // ...
    targets: [
        .plugin(
            name: "FetchConfig",
            capability: .command (
                intent: .custom(verb: "fetch-config", description: "Fetch a `config.json` resource and embed it in the package"),
                permissions: [
                    .allowNetworkConnections(
                        scope: .all(ports: []),
                        reason: "The plugin must connect to a remote server to fetch a resource"
                    ),
                    .writeToPackageDirectory(reason: "Write the fetched `config.json` to the package's root directory")
                ]
            )
        )
    ]
)

Let’s now re-run the plugin and grant it the permissions it needs:

# Set the version of Swift to a development snapshot with Swift 5.9
swiftenv local DEVELOPMENT-SNAPSHOT-2023-02-19-a

# Run the plugin
swift package fetch-config                  \
    --allow-writing-to-package-directory    \
    --allow-network-connections all

After the plugin has finished executing correctly, there should be a new file in the package’s root directory called config.json with the contents of the reponse from https://httpbin.org/json:

Conclusion

Despite that it is still early days in the development of this new feature and it won’t be available until Swift 5.9, I am very excited about the amount of possibilities it will bring to Swift package plugins and the ability to write more complex plugins without having to disable the sandboxed environment they run on.

Having to disable the sandbox environment to make network requests is in fact the issue that sparked the interest for this feature in the first place. The swift-aws-lambda-runtime project has a plugin which needs to interact with Docker to package an executable target into a format suitable for upload to AWS as Swift does not have support for cross compiling code to Linux. This plugin can only be executed by passing the --disable-sandbox flag to the swift package command, which is not ideal as it can have a number of undesired side effects.