Colony

Colony is a self-organizing cluster of nodes on a LAN. Nodes discover each other, gossip membership, and bid for roles: units of work such as storage replication, a password vault, or a web dashboard. That way the cluster keeps running the roles it needs even as nodes join, leave, or die.

How it works

• Discovery & gossip: each node scans the configured IP range for peers and joins the cluster via a gossip protocol (Gossip), building a shared view of membership and health.
• Role assignment: Foreman proposes unclaimed roles to capable nodes. RoleManager/RoleRegistry decide who should run what based on priority, hardware preferences (disk, longevity, etc.) and policy (role_policy in config.json).
• Transport: nodes talk over a TLS listener (Listener/Whisperer) and discover each other over TCP (Discover). A local control socket (ControlSocket) is how colonyctl talks to the node process on the same host.
• Roles shipped today (see role_settings.py):
  • Unit: baseline role every node runs; reports stats/hardware/cluster info.
  • Gossip: membership protocol.
  • Foreman: assigns roles across the cluster.
  • Gateway: relays control commands to nodes that aren't directly reachable.
  • VaultWarden: replicated key/value secret storage.
  • Replicator: content-addressed replicated blob storage used by other roles.
  • Explorer: probes/tracks external hosts outside the discovery range.
  • WebView (on-demand): a small web dashboard (Flask) for the cluster.
  • Chatter (on-demand): peer-to-peer chat between nodes.

Requirements

• Python 3.14
• uv for dependency management

Running a node

uv sync
uv run python main.py

Configuration lives in config.json and can be overridden with COLONY_<SECTION>__<KEY> environment variables (see docker-compose.yml for every available key). At minimum, point discovery.range_start/range_end at the IP range your nodes live on.

Certificates are loaded once per process: restart the node after rotating them. To set a key to null through the environment, use @none (a plain null is read as a string).

Replicated storage (the vault and its replicas) travels as one archive, so it is limited to 16 MiB per node, and to gateway.max_payload_bytes (4 MiB by default) when a replica is fetched through a gateway.

Controlling a node (colonyctl)

colonyctl talks to the local node over its control socket:

./colonyctl.py status              # cluster members and role assignment
./colonyctl.py status --json       # same, as JSON (also used for scripting)
./colonyctl.py role add <Role>     # start an on-demand role on this node
./colonyctl.py webview             # shortcut for `role add WebView`
./colonyctl.py chat <node> [text]  # view/send chat with another node

Docker

docker build -t colony .
docker compose up -d

The container runs on the host network (nodes need to see each other's real IPs for discovery) and persists /app/data, /app/storages, and /app/logs as volumes. TLS cert/key paths are configured via listener/whisperer in config.json.

License

See LICENSE.