Docker Compose Validator Deployment

Deploy a Canton Network validator using Docker Compose

This section describes how to deploy a standalone validator node on a VM or a local machine using Docker Compose. The deployment consists of the validator node along with associated wallet and CNS UIs, and onboards the validator node to the target network.

This deployment is useful for:

• Application development, where one needs an ephemeral validator that is easy to deploy.
• Production validators, with the following caveats:
  • The default deployment is highly insecure. Authentication should be enabled as described in the authentication section.
  • There is no support for ingress from outside your machine, nor is there support for TLS. The deployment should be kept local to your machine only and not exposed externally.
  • Reliability & scalability: docker-compose will restart containers that crash, and the deployment supports backup&restore as detailed below, but a docker-compose deployment is inherently more limited than a cloud-based Kubernetes one.
  • Monitoring: The deployment, as opposed to a Kubernetes-based one, does not include monitoring.
  • For production settings, you should aim to keep your validator up and running constantly, in order to avoid losing out on rewards, and avoid issues with catching up on ledger state after significant downtime.

Requirements

HTTP Proxy configuration

If you need to use an HTTP forward proxy for egress in your environment, you need to set https.proxyHost and https.proxyPort in JAVA_TOOL_OPTIONS in splice-node/docker-compose/validator/compose.yaml to use the HTTP proxy for outgoing connections. You need to do this for both the validator and the participant services:

  services:
    validator:
      environment:
        JAVA_TOOL_OPTIONS: >-
          -Dhttps.proxyHost=your.proxy.host
          -Dhttps.proxyPort=your_proxy_port

  services:
    participant:
      environment:
        JAVA_TOOL_OPTIONS: >-
          -Dhttps.proxyHost=your.proxy.host
          -Dhttps.proxyPort=your_proxy_port

Replace your.proxy.host and your_proxy_port with the actual host and port of your HTTP proxy. Proxy authentication is currently not supported.

Bypassing the proxy for specific hosts

Setting http.nonProxyHosts affects:

• The HTTP client used by the CN apps (Validator, Scan, SV, Wallet).
• JDK-level HTTP clients in the same JVM (via the default ProxySelector). This includes the Auth0 JWK library used by the CN apps and by the Canton participant for JWKS / OIDC discovery, as well as file downloads that use java.net.HttpURLConnection.
• gRPC egress from other components, because gRPC’s Netty transport delegates proxy decisions to the default JDK ProxySelector.

You can set http.nonProxyHosts to bypass the proxy for specific target hosts. Matching hosts will be contacted directly rather than through the configured proxy. This is useful for services that are reachable on the local network, such as an in-cluster Scan instance or internal monitoring endpoints.

The value is a |-separated list of patterns that follows the standard Java nonProxyHosts grammar:

• Patterns match the request host name case-insensitively.
• * is a wildcard. Conventionally it is used at the start (*.internal) or end (10.*) of a pattern.
• Matching is performed on the raw host string from the request URI. No DNS resolution is performed, so localhost and 127.0.0.1 are treated as different names unless you list both.
• An empty value (e.g. -Dhttp.nonProxyHosts=) means “no bypass patterns”.

Example that proxies external traffic from the validator service but bypasses the proxy for localhost / 127.0.0.1, any host in the .internal domain, and any IPv4 address whose literal string representation starts with 10.:

  services:
    validator:
      environment:
        JAVA_TOOL_OPTIONS: >-
          -Dhttps.proxyHost=your.proxy.host
          -Dhttps.proxyPort=your_proxy_port
          -Dhttp.nonProxyHosts=localhost|127.0.0.1|*.internal|10.*

Deployment

Logging into the wallet UI

Docker Compose-based validator deployments use .localhost subdomains for addressing, such as wallet.localhost. .localhost URLs reportedly do not work on some browsers. If you encounter issues please try using a different browser such as Firefox or Chrome. If you’re encountering issues with reaching APIs from a custom program or script, you may need to set the HOST header on HTTP requests explicitly to the target .localhost address.

The wallet UI is accessible at http://wallet.localhost in your browser. The validator administrator’s username is administrator. Insert that name into the username field and click Log in, and you should see the wallet of the administrator of your wallet.

You can also logout of the administrator account and login as any other username. The first time a user logs in, they will be prompted with a message asking them to confirm whether they wish to be onboarded to the validator node.

link to section that explains what this onbarding means

Logging into the CNS UI

You can open your browser at http://ans.localhost (note that this is currently by default ans and not cns), and login using the same administrator user, or any other user that has been onboarded via the wallet, in order to purchase a CNS entry for that user.

Accessing the Canton Participant APIs

The JSON Ledger API is exposed under json-ledger-api.localhost:80. Note that for some clients you may explicitly need to set the Host: json-ledger-api.localhost header for this to get resolved correctly.

The gRPC Ledger API is exposed under grpc-ledger-api.localhost:80. Note that for some clients you may explicitly need to set the :authority: json-ledger-api.localhost pseudo-header for this to get resolved correctly.

The Canton Admin API is not exposed by default as it does not yet support auth. There is a commented out section in nginx.conf that you can enable to expose it if you ensure that it is not exposed publicly, e.g., through network restrictions.

Configuring Authentication

The default deployment uses highly insecure self-signed tokens. Anyone with access to the wallet UI (or the machine and/or its network interface) may log in to your wallet as a user of their choice. For any production use, you should configure proper authentication as described in this section.

Please refer to the authentication section for instructions on how to set up an OAuth provider for your validator. The URLs to configure for callbacks are http://wallet.localhost and http://ans.localhost.

Once you have set up your OAuth provider, you need to configure it by setting the following environment variables in the .env file:

Name	Value
AUTH_URL	The URL of your OIDC provider for obtaining the openid-configuration and jwks.json.
AUTH_JWKS_URL	The URL of your OIDC provider for obtaining the jwks.json, will typically be $/.well-known/jwks.json.
AUTH_WELLKNOWN_URL	The URL of your OIDC provider for obtaining the openid-configuration, will typically be $/.well-known/openid-configuration.
LEDGER_API_AUTH_AUDIENCE	The audience for the participant ledger API, e.g. https://ledger_api.example.com. This sets the ledger-api.auth-services.target-audience configuration for the participant.
LEDGER_API_AUTH_SCOPE	The scope for the participant ledger API. This sets the participant’s ledger-api.auth-services.target-scope configuration. Optional.
VALIDATOR_AUTH_AUDIENCE	The audience for the validator backend API. e.g. https://validator.example.com.
VALIDATOR_AUTH_CLIENT_ID	The client id of the OAuth app for the validator app backend.
VALIDATOR_AUTH_CLIENT_SECRET	The client secret of the OAuth app for the validator app backend.
LEDGER_API_ADMIN_USER	Should match the sub field of JWTs issued for the validator app. For some auth providers, this would be formed as CLIENT_ID@clients.
WALLET_ADMIN_USER	The user ID of the user which should login as the wallet administrator. Note that this should be the full user id, e.g., auth0|43b68e1e4978b000cefba352, not only the suffix 43b68e1e4978b000cefba352.
WALLET_UI_CLIENT_ID	The client id of the OAuth app for the wallet UI.
ANS_UI_CLIENT_ID	The client id of the OAuth app for the CNS UI.
CONTACT_POINT	The contact point for your validator node that can be used by other node operators to reach out to you if needed (slack username or an email address). Optional

In order to enable auth in the deployment, add the -a flag to the start.sh command, as follows:

    ./start.sh -s "<SPONSOR_SV_URL>" -o "<ONBOARDING_SECRET>" -p "<party_hint>" -m "<MIGRATION_ID>" -w -a

If you have already deployed a non-authenticated validator on your machine, you can migrate it to an authenticated one by stopping the validator with ./stop.sh and restarting it with the -a flag as above. The validator operator user will be automatically migrated, and the user indicated by the WALLET_ADMIN_USER variable will be associated with the validator operator party. If you have also onboarded other users onto your validator, those will not be automatically migrated, and you need to manually associate the OAuth users with their corresponding parties. In order to do that, first take note of the party IDs of all relevant users (do this before stopping the unauthenticated validator), e.g. by copying them from the top-right corner of their wallet UIs. Now for every user that you wish to migrate, follow the instructions for associating a user with a party in the Users, Parties and Wallets in the Splice Wallet section, but replace the admin party ID with the party ID which you wish to associate with each user.

Configuring Automatic Traffic Purchases

Your node is configured to automatically purchase traffic on a pay-as-you-go basis (see automatically purchase traffic). To tune to your needs, you can set environment variables, for example:

   export TARGET_TRAFFIC_THROUGHPUT=20000 # target throughput in bytes/second
   export MIN_TRAFFIC_TOPUP_INTERVAL="1m" # minimum interval between top-ups

On each successful top-up, the validator app purchases a top-up amount of roughly targetThroughput * minTopupInterval bytes of traffic (specific amount can vary due to rounding-up). The minTopupInterval allows validator operators to control the upper-bound frequency at which automated top-ups happen. If the top-up amount is below the synchronizer-wide minTopupAmount (see traffic parameters), minTopupInterval is automatically stretched so that at least minTopupAmount bytes of traffic are purchased while respecting the configured targetThroughput.

The next top-up gets triggered when all of the following conditions are met:

• The available extra traffic balance drops below the configured top-up amount (i.e., below targetThroughput * minTopupInterval).
• At least minTopupInterval has elapsed since the last top-up.
• The validator has sufficient CC in its wallet to buy the top-up amount worth on traffic (except on DevNet, where the validator app will automatically tap enough coin to purchase traffic).

Validators receive a small amount of free traffic from the Super Validators, which suffices for submitting the top-up transaction. However, if many other transactions are submitted, you may run into a situation where you have exhausted also the free traffic, thus the validator cannot submit the top-up transaction. The free traffic grant accumulates gradually and continuously. When no transactions are submitted, it takes about twenty minutes for free traffic to accumulate to the maximum possible. If you’ve consumed your traffic balance by submitting too many transactions without purchasing traffic, pause your Validator node (validator app and participant) for twenty minutes to allow your free traffic balance to accumulate.

Configuring sweeps and auto-accepts of transfer offers

You can optionally configure the validator to automatically create transfer offers to other parties on the network whenever the balance of certain parties that it hosts exceeds a certain threshold.

Whenever the balance of <senderPartyID> exceeds maxBalanceUSD, the validator will automatically create a transfer offer to <receiverPartyId>, for an amount that leaves minBalanceUSD in the sender’s wallet. Note that you will need to know the party IDs of both the sender and receiver, which can be copied from the wallet UIs of the respective users (in the top right corner). This therefore needs to be applied to the Helm chart in a second step after the initial deployment, once the party IDs are known.

Whenever the validator receives a transfer offer from <senderPartyID> to <receiverPartyId>, it will automatically accept it. Similarly to sweeps, party IDs must be known in order to apply this configuration.

To do so, fill the following section and add the following additional config to your validator environment:

# sweep by transferring directly through the transfer preapproval of the receiver,
# if set to false sweeping creates transfer offers that need to be accepted on the receiver side.
# Note that this refers to the preapprovals described in /appdev/modules/m7-canton-coin-preapprovals
# and not to auto accepting transfers. Auto accept transfers does not setup preapproval contracts that allow
# for a direct transfer but just automates the acceptance of the transfer offer so in that case
# useTransferPreapproval should be set to false.
services:
  validator:
    environment:
       - |
          ADDITIONAL_CONFIG_WALLET_SWEEP=
            canton.validator-apps.validator_backend.wallet-sweep {
              "<senderPartyId1>" {
                max-balance-usd = 1000
                min-balance-usd = 100
                receiver = "<receiverPartyId>"
                use-transfer-preapproval = false
              }
            }

Similarly, you can configure the validator to automatically accept transfer offers from certain parties on the network. To do so, add the following additional config:

   services:
     validator:
       environment:
          - |
             ADDITIONAL_CONFIG_AUTO_ACCEPT_TRANSFERS=
               canton.validator-apps.validator_backend.auto-accept-transfers {
                 "<receiverPartyId>" {
                   from-parties = ["<senderPartyId1>", "<senderPartyId2>"]
                 }
               }

Integration with systemd and other init systems

If you want to manage the validator through systemd or a similar init system, create a service that calls the start.sh script with the right arguments. However, note that start.sh invokes docker compose up with the -d/--detach option so the script exits after the containers are up instead of continuing running.

You need to make sure that your service does not stop docker compose at that point. To accomplish this with systemd set RemainAfterExit=true. Refer to the systemd documentation for more details. If you are using another init system, look for similar options to ensure that docker compose continues running after the script exits.

Alternatively, you can edit the script to remove the -d option so the script continues running.