Colony

Colony is a self-organizing cluster of nodes on a LAN. Nodes discover each other, gossip membership, and bid for roles: units of work such as storage replication, a password vault, or a web dashboard. That way the cluster keeps running the roles it needs even as nodes join, leave, or die.

How it works

Discovery & gossip: each node scans the configured IP range for peers and joins the cluster via a gossip protocol (Gossip), building a shared view of membership and health.
Role assignment: Foreman proposes unclaimed roles to capable nodes. RoleManager/RoleRegistry decide who should run what based on priority, hardware preferences (disk, longevity, etc.) and policy (role_policy in config.json).
Transport: nodes talk over a TLS listener (Listener/Whisperer) and discover each other over UDP (Discover). A local control socket (ControlSocket) is how colonyctl talks to the node process on the same host.
Roles shipped today (see role_settings.py):
- Unit: baseline role every node runs; reports stats/hardware/cluster info.
- Gossip: membership protocol.
- Foreman: assigns roles across the cluster.
- Gateway: relays control commands to nodes that aren't directly reachable.
- VaultWarden: replicated key/value secret storage.
- Replicator: content-addressed replicated blob storage used by other roles.
- Explorer: probes/tracks external hosts outside the discovery range.
- WebView (on-demand): a small web dashboard (Flask) for the cluster.
- Chatter (on-demand): peer-to-peer chat between nodes.

Requirements

Python 3.14
uv for dependency management

Running a node

uv sync
uv run python main.py

Configuration lives in config.json and can be overridden with COLONY_<SECTION>__<KEY> environment variables (see docker-compose.yml for every available key). At minimum, point discovery.range_start/range_end at the IP range your nodes live on.

Controlling a node (`colonyctl`)

colonyctl talks to the local node over its control socket:

./colonyctl.py status              # cluster members and role assignment
./colonyctl.py status --json       # same, as JSON (also used for scripting)
./colonyctl.py role add <Role>     # start an on-demand role on this node
./colonyctl.py webview             # shortcut for `role add WebView`
./colonyctl.py chat <node> [text]  # view/send chat with another node

Docker

docker build -t colony .
docker compose up -d

The container runs on the host network (nodes need to see each other's real IPs for discovery) and persists /app/data, /app/storages, and /app/logs as volumes. TLS cert/key paths are configured via listener/whisperer in config.json.

License

See LICENSE.

2.8 KiB Raw Blame History