Cluster Agent

Deployment

Docker compose for a local 3-node cluster, Kubernetes manifests for production, and the bare-metal pattern for institutional setups.

Deployment

The MPC Agent ships as a single binary. The supported deployment patterns are:

  • Docker compose — the canonical local-dev setup. Three nodes plus a message broker in one stack. Same image runs in CI and on operator laptops.
  • Kubernetes — production. One StatefulSet per node, a shared broker, persistent volumes for share state, Service / Ingress for the HTTPS interface.
  • Bare metal — institutional setups behind a security perimeter. Three hosts, one binary each, a fourth host (or a managed broker) for intra-cluster transport.

This page covers the first two end-to-end. Bare metal is described as a recipe at the bottom.


Docker compose — local 3-node cluster

The simplest way to run a cluster on your laptop. Boots three agents on ports 3001 / 3002 / 3003, a broker for intra-cluster transport, and shared volumes for each node's state.

:::note Three nodes is the recommended default, but the cluster works at other sizes. For a two-node cluster, drop node-3 from the stack and run a 2-of-2 cohort (threshold: 2, maxShares: 2, node indices 0 and 1); larger cohorts (5, 7, …) work the same way with a higher maxShares. Whatever the size, every node must be listed in each ceremony's peers[] and maxShares must equal the node count. The host ports 30013003 are arbitrary — change them if they're already taken on your machine. :::

Prerequisites:

  • Docker Engine 20.10+ with docker compose.
  • A license certificate (request from Zafeguard; for local-only testing you can opt into the embedded development key).

Layout:

# docker-compose.yml — minimal 3-node setup.
services:
  broker:
    image: rabbitmq:3.13-management-alpine
    hostname: broker
    environment:
      RABBITMQ_DEFAULT_USER: mpc
      RABBITMQ_DEFAULT_PASS: mpc
    expose: ["5672"]
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "check_port_connectivity"]
      interval: 10s
      retries: 10

  node-1:
    image: zafeguard/mpc-agent:latest
    container_name: node-1
    depends_on:
      broker: { condition: service_healthy }
    ports: ["3001:3000"]
    volumes:
      - node-1-data:/app/data
      - ./config/node-1.yml:/app/config.yml:ro
    environment:
      NODE_PRIVATE_KEY: ${NODE_1_PRIVATE_KEY}
      MPC_LICENSE_CERT: ${MPC_LICENSE_CERT}
      API_KEY_1: ${API_KEY_1}

  node-2:
    image: zafeguard/mpc-agent:latest
    container_name: node-2
    depends_on:
      broker: { condition: service_healthy }
    ports: ["3002:3000"]
    volumes:
      - node-2-data:/app/data
      - ./config/node-2.yml:/app/config.yml:ro
    environment:
      NODE_PRIVATE_KEY: ${NODE_2_PRIVATE_KEY}
      MPC_LICENSE_CERT: ${MPC_LICENSE_CERT}
      API_KEY_1: ${API_KEY_1}

  node-3:
    image: zafeguard/mpc-agent:latest
    container_name: node-3
    depends_on:
      broker: { condition: service_healthy }
    ports: ["3003:3000"]
    volumes:
      - node-3-data:/app/data
      - ./config/node-3.yml:/app/config.yml:ro
    environment:
      NODE_PRIVATE_KEY: ${NODE_3_PRIVATE_KEY}
      MPC_LICENSE_CERT: ${MPC_LICENSE_CERT}
      API_KEY_1: ${API_KEY_1}

volumes:
  node-1-data:
  node-2-data:
  node-3-data:

Per-node config (config/node-1.yml, etc. — same shape, different node.id):

node:
  id: "node-1"
  display_name: "MPC Node 1"

identity:
  private_key_hex: "${NODE_PRIVATE_KEY}"
  allow_dynamic_nodes: false
  trusted_peers:                      # include EVERY node, this one too (or leave [] to disable pinning)
    - "<node-1-public-key-hex>"
    - "<node-2-public-key-hex>"
    - "<node-3-public-key-hex>"

license:
  certificate: "${MPC_LICENSE_CERT}"

rabbitmq:
  url: "amqp://mpc:mpc@broker:5672/%2F"

database:
  url: "sqlite:///app/data/agent.db?mode=rwc"

api:
  host: "0.0.0.0"
  port: 3000
  auth:
    auth_type: api_key
    api_keys:
      - "${API_KEY_1}"
  rate_limit:
    enabled: true
    max_per_minute: 300

Generate node keypairs and bring the stack up:

# 1) Generate one Ed25519 keypair per node. gen-secret prints two
#    labelled lines: "Private key : <hex>" and "Public key  : <hex>".
docker run --rm zafeguard/mpc-agent:latest gen-secret > node-1.key
docker run --rm zafeguard/mpc-agent:latest gen-secret > node-2.key
docker run --rm zafeguard/mpc-agent:latest gen-secret > node-3.key

# 2) Helpers to pull each half out of the two-line output.
priv() { awk '/Private key/ {print $NF}' "$1"; }
pub()  { awk '/Public key/  {print $NF}' "$1"; }

# 3) Fill in .env with the private seeds, license token, and an API key.
#    The license is a single-line token file (not a PEM), e.g. license.txt.
cat > .env <<EOF
NODE_1_PRIVATE_KEY=$(priv node-1.key)
NODE_2_PRIVATE_KEY=$(priv node-2.key)
NODE_3_PRIVATE_KEY=$(priv node-3.key)
MPC_LICENSE_CERT=$(cat license.txt)
API_KEY_1=$(openssl rand -hex 32)
EOF

# 4) Put EVERY node's PUBLIC key — including each node's own — into the
#    trusted_peers list of every node config. Print them with:
echo "node-1 pubkey: $(pub node-1.key)"
echo "node-2 pubkey: $(pub node-2.key)"
echo "node-3 pubkey: $(pub node-3.key)"

# 5) Start the cluster.
docker compose up -d

# 6) Verify each node is healthy.
curl http://localhost:3001/health
curl http://localhost:3002/health
curl http://localhost:3003/health
# Each should return {"status":"ok"}.

Once the cluster is up, point an EmbeddedAgent at the three nodes:

const signer = new EmbeddedAgent({
  agents: [
    { baseUrl: 'http://localhost:3001', apiKey: process.env.API_KEY_1! },
    { baseUrl: 'http://localhost:3002', apiKey: process.env.API_KEY_1! },
    { baseUrl: 'http://localhost:3003', apiKey: process.env.API_KEY_1! },
  ],
  threshold: 2,
  curve: Curve.Secp256k1,
});

Kubernetes — production

For production, run each node as a StatefulSet so it gets a stable network identity and a persistent volume. The intra-cluster broker goes in its own StatefulSet (or use a managed broker like AWS MQ / CloudAMQP).

Per-node StatefulSet (one per node — node-1, node-2, node-3):

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: node-1
spec:
  serviceName: node-1
  replicas: 1
  selector:
    matchLabels: { app: node-1 }
  template:
    metadata:
      labels: { app: node-1 }
    spec:
      # The image runs as a non-root user (uid 65532). fsGroup makes the
      # persistent volume group-writable so the agent can create its
      # database file; without it the volume is root-owned and writes fail.
      securityContext:
        fsGroup: 65532
      containers:
        - name: agent
          image: zafeguard/mpc-agent:latest
          # The default entrypoint looks for /app/config.yml; point it at the
          # mounted config explicitly.
          args: ["start", "--config", "/etc/mpc-agent/config.yml"]
          ports:
            - containerPort: 3000
              name: http
          env:
            - name: NODE_PRIVATE_KEY
              valueFrom:
                secretKeyRef: { name: node-1-secrets, key: private-key }
            - name: MPC_LICENSE_CERT
              valueFrom:
                secretKeyRef: { name: mpc-license, key: certificate }
            - name: API_KEY_1
              valueFrom:
                secretKeyRef: { name: mpc-api-keys, key: key-1 }
          volumeMounts:
            - name: data
              mountPath: /app/data
            - name: config
              mountPath: /etc/mpc-agent
              readOnly: true
          readinessProbe:
            httpGet: { path: /ready, port: 3000 }
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            httpGet: { path: /health, port: 3000 }
            initialDelaySeconds: 30
            periodSeconds: 30
      volumes:
        - name: config
          configMap: { name: node-1-config }
  volumeClaimTemplates:
    - metadata: { name: data }
      spec:
        accessModes: [ReadWriteOnce]
        resources: { requests: { storage: 10Gi } }
        storageClassName: gp3  # AWS — pick your backing

Service per node (each node has its own internal service for the StatefulSet's stable DNS):

apiVersion: v1
kind: Service
metadata:
  name: node-1
spec:
  selector: { app: node-1 }
  ports:
    - port: 3000
      targetPort: http

Ingress — terminate TLS at the ingress controller (ACM + ALB on EKS, Cert Manager + nginx-ingress on bare K8s):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mpc-cluster
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:...
spec:
  ingressClassName: alb
  rules:
    - host: node-1.mpc.yourcompany.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend: { service: { name: node-1, port: { number: 3000 } } }
    - host: node-2.mpc.yourcompany.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend: { service: { name: node-2, port: { number: 3000 } } }
    - host: node-3.mpc.yourcompany.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend: { service: { name: node-3, port: { number: 3000 } } }

Intra-cluster broker — either the official AMQP-broker Helm chart or a managed broker. Set rabbitmq.url in each agent's config to the broker's AMQP URL. A 3-replica broker handles most production loads; for very high volume consider a dedicated managed broker.

Resource sizing per scaling tier

The manifest above is generic. For tier-tuned resource requests / limits + readiness gates matching the configuration.md scaling-tier table, drop the relevant block into the Deployment / StatefulSet spec.template.spec.containers[0]:

Production-Small (up to 10k ceremonies/day)

resources:
  requests:
    cpu: 500m
    memory: 1Gi
  limits:
    cpu: 1500m
    memory: 1.5Gi
readinessProbe:
  httpGet: { path: /ready, port: 3000 }
  initialDelaySeconds: 5
  periodSeconds: 10
  failureThreshold: 3
livenessProbe:
  httpGet: { path: /health, port: 3000 }
  initialDelaySeconds: 30
  periodSeconds: 30

No HPA needed at this tier — one replica per cluster node is enough.

Production-Medium (up to 100k ceremonies/day)

resources:
  requests:
    cpu: 2
    memory: 4Gi
  limits:
    cpu: 4
    memory: 5Gi
readinessProbe:
  httpGet: { path: /ready, port: 3000 }
  initialDelaySeconds: 5
  periodSeconds: 10
livenessProbe:
  httpGet: { path: /health, port: 3000 }
  initialDelaySeconds: 30
  periodSeconds: 30

PodDisruptionBudget keeps threshold-many replicas available during planned maintenance (node drains, version upgrades):

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: mpc-agent
spec:
  minAvailable: 2  # for a 3-node cluster running 2-of-3 cohort
  selector:
    matchLabels:
      app.kubernetes.io/component: mpc-agent

Production-Large (up to 1M ceremonies/day)

resources:
  requests:
    cpu: 4
    memory: 16Gi
  limits:
    cpu: 8
    memory: 20Gi
readinessProbe:
  httpGet: { path: /ready, port: 3000 }
  initialDelaySeconds: 10
  periodSeconds: 5
  failureThreshold: 2  # tighter — pull traffic faster at this scale
livenessProbe:
  httpGet: { path: /health, port: 3000 }
  initialDelaySeconds: 60
  periodSeconds: 30

At this tier the broker is the scaling bottleneck, not the agent — run a 3-replica AMQP broker cluster (Helm chart) or a managed broker (AWS MQ Multi-AZ, CloudAMQP "Production" plan). The agent's rabbitmq.publish_channels and workers config knobs (per configuration.md) absorb the higher volume; nothing else changes at the manifest layer.

Enterprise XL (10M+ ceremonies/day)

resources:
  requests:
    cpu: 8
    memory: 64Gi
  limits:
    cpu: 16
    memory: 80Gi
readinessProbe:
  httpGet: { path: /ready, port: 3000 }
  initialDelaySeconds: 15
  periodSeconds: 5
  failureThreshold: 2
livenessProbe:
  httpGet: { path: /health, port: 3000 }
  initialDelaySeconds: 60
  periodSeconds: 30

At this tier the cluster runs against a managed Postgres or RDS instance (the docs flag SQLite-to-Postgres migration at this scale); the matching database.max_connections: 100 in the agent config requires a Postgres instance sized for at least 3× that (replica_count × max_connections).

/ready semantics for rolling deploys

The /ready probe returns 503 when ANY of db / rabbitmq / license is failing. K8s rolling deploys + load balancers use this to drain traffic away from a pod before it starts rejecting ceremonies — especially important at higher tiers where the cost of a single 5xx burst is hundreds of failed user requests.

What flips /ready red:

  • Database unreachable (sqlx pool exhausted, or postgres down) — db: false
  • Broker disconnect (AMQP TCP socket closed) — broker: false
  • License expired (cert expiry timestamp passed) — license: false

The license: false case is the under-appreciated one — a cert that expires mid-deploy would otherwise silently start failing every ceremony with a 403, and the only signal would be the ceremony_aborted_total rate climbing in /metrics. The readiness check drains traffic to the pod first, giving operators time to renew + roll out a new cert without user-visible impact.


Bare-metal pattern

For institutional setups behind a security perimeter (often required by compliance — HSM-backed key storage, network isolation, no container runtime).

  • Three hosts, one MPC agent binary per host. Use systemd to manage the process.
  • One broker host running an AMQP broker (or a managed AWS MQ instance).
  • One TLS terminator — typically nginx on each agent host, but ALB / HAProxy / Cloudflare all work.
  • Persistent volume per node — RAID-mirrored disk recommended; share state must survive a single-disk failure. Point the config's database.url (SQLite file) at a path inside this directory — e.g. sqlite:////var/lib/mpc-agent/agent.db?mode=rwc — so it's covered by the unit's ReadWritePaths.

The agent binary is statically linked (musl build available) so it runs on any modern Linux without per-host toolchains. Drop the binary at /usr/local/bin/agent, the config at /etc/mpc-agent/config.yml, the state directory at /var/lib/mpc-agent, and the unit file at /etc/systemd/system/mpc-agent.service:

[Unit]
Description=Zafeguard MPC Agent
After=network.target
Requires=network.target

[Service]
Type=simple
User=mpc-agent
Group=mpc-agent
EnvironmentFile=/etc/mpc-agent/agent.env
ExecStart=/usr/local/bin/agent start --config /etc/mpc-agent/config.yml
Restart=on-failure
RestartSec=5s
ProtectSystem=strict
ReadWritePaths=/var/lib/mpc-agent
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

/etc/mpc-agent/agent.env holds the env vars (NODE_PRIVATE_KEY, MPC_LICENSE_CERT, API_KEY_1) with chmod 0400.


Backups

The state directory (/var/lib/mpc-agent in the examples above) holds:

  • The MPC shares. Lose these → that node can no longer participate in signing. If you lose the shares on threshold-many nodes, every wallet in the cohort is unrecoverable except via cold recovery.
  • Per-session DB rows. Active and recently-completed ceremony state. Loss only affects sessions that were in flight — durable signers (those whose state was already persisted) survive.

Back the volume up regularly. The contents are ciphertext-at-rest (sealed by the node's identity key), so a snapshot-level backup is safe to ship off-host.


Next

  • Configuration → — every config knob explained: identity, license, intra-cluster transport, storage, API auth, rate limits, policy webhooks, performance and limits.
  • How it works → — cluster topology, ceremony coordination, identity and trust.
  • Integration patterns → — Embedded Agent setup, custody-backend setup, hybrid.

On this page