yadunut/clan-core
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Add wireguard service module

Browse Source
This commit is contained in:
pinpox
2025-04-18 13:09:48 +02:00
committed by Jörg Thalheim
parent 88871bea69
commit 414952dfa3
53 changed files with 1166 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

217
clanServices/wireguard/README.md Normal file
View File

@@ -0,0 +1,217 @@
# Wireguard VPN Service
This service provides a Wireguard-based VPN mesh network with automatic IPv6 address allocation and routing between clan machines.
## Overview
The wireguard service creates a secure mesh network between clan machines using two roles:
- **Controllers**: Machines with public endpoints that act as connection points and routers
- **Peers**: Machines that connect through controllers to access the network
## Features
- Automatic IPv6 address allocation using ULA (Unique Local Address) prefixes
- Full mesh connectivity between all machines
- Automatic key generation and distribution
- IPv6 forwarding on controllers for inter-peer communication
- Support for multiple controllers for redundancy
## Network Architecture
### IPv6 Address Allocation
- Base network: `/40` ULA prefix (deterministically generated from instance name)
- Controllers: Each gets a `/56` subnet from the base `/40`
- Peers: Each gets a unique 64-bit host suffix that is used in ALL controller subnets
### Addressing Design
- Each peer generates a unique host suffix (e.g., `:8750:a09b:0:1`)
- This suffix is appended to each controller's `/56` prefix to create unique addresses
- Example: peer1 with suffix `:8750:a09b:0:1` gets:
- `fd51:19c1:3b:f700:8750:a09b:0:1` in controller1's subnet
- `fd51:19c1:c1:aa00:8750:a09b:0:1` in controller2's subnet
- Controllers allow each peer's `/96` subnet for routing flexibility
### Connectivity
- Peers use a single WireGuard interface with multiple IPs (one per controller subnet)
- Controllers connect to ALL other controllers and ALL peers on a single interface
- Controllers have IPv6 forwarding enabled to route traffic between peers
- All traffic between peers flows through controllers
- Symmetric routing is maintained as each peer has consistent IPs across all controllers
### Example Network Topology
```mermaid
graph TB
subgraph Controllers
C1[controller1<br/>endpoint: vpn1.example.com<br/>fd51:19c1:3b:f700::/56]
C2[controller2<br/>endpoint: vpn2.example.com<br/>fd51:19c1:c1:aa00::/56]
end
subgraph Peers
P1[peer1<br/>designated: controller1]
P2[peer2<br/>designated: controller2]
P3[peer3<br/>designated: controller1]
end
%% Controllers connect to each other
C1 <--> C2
%% All peers connect to all controllers
P1 <--> C1
P1 <--> C2
P2 <--> C1
P2 <--> C2
P3 <--> C1
P3 <--> C2
%% Peer-to-peer traffic flows through controllers
P1 -.->|via controllers| P3
P1 -.->|via controllers| P2
P2 -.->|via controllers| P3
classDef controller fill:#f9f,stroke:#333,stroke-width:4px
classDef peer fill:#bbf,stroke:#333,stroke-width:2px
class C1,C2 controller
class P1,P2,P3 peer
```
## Configuration
### Basic Setup with Single Controller
```nix
# In your flake.nix or inventory
{
services.wireguard.server1 = {
roles.controller = {
# Public endpoint where this controller can be reached
endpoint = "vpn.example.com";
# Optional: Change the UDP port (default: 51820)
port = 51820;
};
};
services.wireguard.laptop1 = {
roles.peer = {
# No configuration needed if only one controller exists
};
};
}
```
### Multiple Controllers Setup
```nix
{
services.wireguard.server1 = {
roles.controller = {
endpoint = "vpn1.example.com";
};
};
services.wireguard.server2 = {
roles.controller = {
endpoint = "vpn2.example.com";
};
};
services.wireguard.laptop1 = {
roles.peer = {
# Must specify which controller's subnet to use for IP allocation
controller = "server1";
};
};
}
```
### Advanced Options
### Automatic Hostname Resolution
The wireguard service automatically adds entries to `/etc/hosts` for all machines in the network. Each machine is accessible via its hostname in the format `<machine-name>.<instance-name>`.
For example, with an instance named `vpn`:
- `server1.vpn` - resolves to server1's IPv6 address
- `laptop1.vpn` - resolves to laptop1's IPv6 address
This allows machines to communicate using hostnames instead of IPv6 addresses:
```bash
# Ping another machine by hostname
ping6 server1.vpn
# SSH to another machine
ssh user@laptop1.vpn
```
## Troubleshooting
### Check Wireguard Status
```bash
sudo wg show
```
### Verify IP Addresses
```bash
ip addr show dev <instance-name>
```
### Check Routing
```bash
ip -6 route show dev <instance-name>
```
### Interface Fails to Start: "Address already in use"
If you see this error in your logs:
```
wireguard: Could not bring up interface, ignoring: Address already in use
```
This means the configured port (default: 51820) is already in use by another service or wireguard instance. Solutions:
1. **Check for conflicting wireguard instances:**
```bash
sudo wg show
sudo ss -ulnp | grep 51820
```
2. **Use a different port:**
```nix
services.wireguard.myinstance = {
roles.controller = {
endpoint = "vpn.example.com";
port = 51821; # Use a different port
};
};
```
3. **Ensure unique ports across multiple instances:**
If you have multiple wireguard instances on the same machine, each must use a different port.
### Key Management
Keys are automatically generated and stored in the clan vars system. To regenerate keys:
```bash
# Regenerate keys for a specific machine and instance
clan vars generate --service wireguard-keys-<instance-name> --regenerate --machine <machine-name>
# Apply the new keys
clan machines update <machine-name>
```
## Security Considerations
- All traffic is encrypted using Wireguard's modern cryptography
- Private keys never leave the machines they're generated on
- Public keys are distributed through the clan vars system
- Controllers must have publicly accessible endpoints
- Firewall rules are automatically configured for the Wireguard ports
## Requirements
- Controllers must have a publicly accessible endpoint (domain name or static IP)
- IPv6 support in the kernel (standard in modern systems)
- UDP port access (default: 51820, configurable)

456
clanServices/wireguard/default.nix Normal file
View File

@@ -0,0 +1,456 @@
/*
There are two roles: peers and controllers:
- Every controller has an endpoint set
- There can be multiple peers
- There has to be one or more controllers
- Peers connect to ALL controllers (full mesh)
- If only one controller exists, peers automatically use it for IP allocation
- If multiple controllers exist, peers must specify which controller's subnet to use
- Controllers have IPv6 forwarding enabled, every peer and controller can reach
everyone else, via extra controller hops if necessary
Example:
┌───────────────────────────────┐
│ ◄───────────── │
│ controller2 controller1
│ ▲ ─────────────► ▲ ▲
│ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │
│ │ │ │ └───────────────┐ │ │ │ │
│ │ │ └──────────────┐ │ │ │ │ │
│ ▼ │ ▼ ▼ ▼
└─► peer2 │ peer1 peer3
│ ▲
└──────────┘
Network Architecture:
IPv6 Address Allocation:
- Base network: /40 ULA prefix (generated from instance name)
- Controllers: Each gets a /56 subnet from the base /40
- Peers: Each gets a unique host suffix that is used in ALL controller subnets
Address Assignment:
- Each peer generates a unique 64-bit host suffix (e.g., :8750:a09b:0:1)
- This suffix is appended to each controller's /56 prefix
- Example: peer1 with suffix :8750:a09b:0:1 gets:
- fd51:19c1:3b:f700:8750:a09b:0:1 in controller1's subnet
- fd51:19c1:c1:aa00:8750:a09b:0:1 in controller2's subnet
Peers: Use a SINGLE interface that:
- Connects to ALL controllers
- Has multiple IPs, one in each controller's subnet (with /56 prefix)
- Routes to each controller's /56 subnet via that controller
- allowedIPs: Each controller's /56 subnet
- No routing conflicts due to unique IPs per subnet
Controllers: Use a SINGLE interface that:
- Connects to ALL peers and ALL other controllers
- Gets a /56 subnet from the base /40 network
- Has IPv6 forwarding enabled for routing between peers
- allowedIPs:
- For peers: A /96 range containing the peer's address in this controller's subnet
- For other controllers: The controller's /56 subnet
*/
{ ... }:
let
# Shared module for extraHosts configuration
extraHostsModule =
{
instanceName,
settings,
roles,
config,
lib,
...
}:
{
networking.extraHosts =
let
domain = if settings.domain == null then instanceName else settings.domain;
# Controllers use their subnet's ::1 address
controllerHosts = lib.mapAttrsToList (
name: _value:
let
prefix = builtins.readFile (
config.clan.core.settings.directory
+ "/vars/per-machine/${name}/wireguard-network-${instanceName}/prefix/value"
);
# Controller IP is always ::1 in their subnet
ip = prefix + "::1";
in
"${ip} ${name}.${domain}"
) roles.controller.machines;
# Peers use their suffix in their designated controller's subnet only
peerHosts = lib.mapAttrsToList (
peerName: peerValue:
let
peerSuffix = builtins.readFile (
config.clan.core.settings.directory
+ "/vars/per-machine/${peerName}/wireguard-network-${instanceName}/suffix/value"
);
# Determine designated controller
designatedController =
if (builtins.length (builtins.attrNames roles.controller.machines) == 1) then
(builtins.head (builtins.attrNames roles.controller.machines))
else
peerValue.settings.controller;
controllerPrefix = builtins.readFile (
config.clan.core.settings.directory
+ "/vars/per-machine/${designatedController}/wireguard-network-${instanceName}/prefix/value"
);
peerIP = controllerPrefix + ":" + peerSuffix;
in
"${peerIP} ${peerName}.${domain}"
) roles.peer.machines;
in
builtins.concatStringsSep "\n" (controllerHosts ++ peerHosts);
};
# Shared interface options
sharedInterface =
{ lib, ... }:
{
options.port = lib.mkOption {
type = lib.types.int;
example = 51820;
default = 51820;
description = ''
Port for the wireguard interface
'';
};
options.domain = lib.mkOption {
type = lib.types.nullOr lib.types.str;
defaultText = lib.literalExpression "instanceName";
default = null;
description = ''
Domain suffix to use for hostnames in /etc/hosts.
Defaults to the instance name.
'';
};
};
in
{
_class = "clan.service";
manifest.name = "clan-core/wireguard";
manifest.description = "Wireguard-based VPN mesh network with automatic IPv6 address allocation";
manifest.categories = [
"System"
"Network"
];
manifest.readme = builtins.readFile ./README.md;
# Peer options and configuration
roles.peer = {
interface =
{ lib, ... }:
{
imports = [ sharedInterface ];
options.controller = lib.mkOption {
type = lib.types.str;
example = "controller1";
description = ''
Machinename of the controller to attach to
'';
};
};
perInstance =
{
instanceName,
settings,
roles,
machine,
...
}:
{
# Set default domain to instanceName
# Peers connect to all controllers
nixosModule =
{
config,
pkgs,
lib,
...
}:
{
imports = [
(extraHostsModule {
inherit
instanceName
settings
roles
config
lib
;
})
];
# Network allocation generator for this peer - generates host suffix
clan.core.vars.generators."wireguard-network-${instanceName}" = {
files.suffix.secret = false;
runtimeInputs = with pkgs; [
python3
];
# Invalidate on hostname changes
validation.hostname = machine.name;
script = ''
${pkgs.python3}/bin/python3 ${./ipv6_allocator.py} "$out" "${instanceName}" peer "${machine.name}"
'';
};
# Single wireguard interface with multiple IPs
networking.wireguard.interfaces."${instanceName}" = {
ips =
# Get this peer's suffix
let
peerSuffix =
config.clan.core.vars.generators."wireguard-network-${instanceName}".files.suffix.value;
in
# Create an IP in each controller's subnet
lib.mapAttrsToList (
ctrlName: _:
let
controllerPrefix = builtins.readFile (
config.clan.core.settings.directory
+ "/vars/per-machine/${ctrlName}/wireguard-network-${instanceName}/prefix/value"
);
peerIP = controllerPrefix + ":" + peerSuffix;
in
"${peerIP}/56"
) roles.controller.machines;
privateKeyFile =
config.clan.core.vars.generators."wireguard-keys-${instanceName}".files."privatekey".path;
# Connect to all controllers
peers = lib.mapAttrsToList (name: value: {
publicKey = (
builtins.readFile (
config.clan.core.settings.directory
+ "/vars/per-machine/${name}/wireguard-keys-${instanceName}/publickey/value"
)
);
# Allow each controller's /56 subnet
allowedIPs = [
"${
builtins.readFile (
config.clan.core.settings.directory
+ "/vars/per-machine/${name}/wireguard-network-${instanceName}/prefix/value"
)
}::/56"
];
endpoint = "${value.settings.endpoint}:${toString value.settings.port}";
persistentKeepalive = 25;
}) roles.controller.machines;
};
};
};
};
# Controller options and configuration
roles.controller = {
interface =
{ lib, ... }:
{
imports = [ sharedInterface ];
options.endpoint = lib.mkOption {
type = lib.types.str;
example = "vpn.clan.lol";
description = ''
Endpoint where the controller can be reached
'';
};
};
perInstance =
{
settings,
instanceName,
roles,
machine,
...
}:
{
# Controllers connect to all peers and other controllers
nixosModule =
{
config,
pkgs,
lib,
...
}:
let
allOtherControllers = lib.filterAttrs (name: _v: name != machine.name) roles.controller.machines;
allPeers = roles.peer.machines;
in
{
imports = [
(extraHostsModule {
inherit
instanceName
settings
roles
config
lib
;
})
];
# Network allocation generator for this controller
clan.core.vars.generators."wireguard-network-${instanceName}" = {
files.prefix.secret = false;
runtimeInputs = with pkgs; [
python3
];
# Invalidate on network or hostname changes
validation.hostname = machine.name;
script = ''
${pkgs.python3}/bin/python3 ${./ipv6_allocator.py} "$out" "${instanceName}" controller "${machine.name}"
'';
};
# Enable ip forwarding, so wireguard peers can reach eachother
boot.kernel.sysctl."net.ipv6.conf.all.forwarding" = 1;
networking.firewall.allowedUDPPorts = [ settings.port ];
# Single wireguard interface
networking.wireguard.interfaces."${instanceName}" = {
listenPort = settings.port;
ips = [
# Controller uses ::1 in its /56 subnet but with /40 prefix for proper routing
"${config.clan.core.vars.generators."wireguard-network-${instanceName}".files.prefix.value}::1/40"
];
privateKeyFile =
config.clan.core.vars.generators."wireguard-keys-${instanceName}".files."privatekey".path;
# Connect to all peers and other controllers
peers = lib.mapAttrsToList (
name: value:
if allPeers ? ${name} then
# For peers: they now have our entire /56 subnet
{
publicKey = (
builtins.readFile (
config.clan.core.settings.directory
+ "/vars/per-machine/${name}/wireguard-keys-${instanceName}/publickey/value"
)
);
# Allow the peer's /96 range in ALL controller subnets
allowedIPs = lib.mapAttrsToList (
ctrlName: _:
let
controllerPrefix = builtins.readFile (
config.clan.core.settings.directory
+ "/vars/per-machine/${ctrlName}/wireguard-network-${instanceName}/prefix/value"
);
peerSuffix = builtins.readFile (
config.clan.core.settings.directory
+ "/vars/per-machine/${name}/wireguard-network-${instanceName}/suffix/value"
);
in
"${controllerPrefix}:${peerSuffix}/96"
) roles.controller.machines;
persistentKeepalive = 25;
}
else
# For other controllers: use their /56 subnet
{
publicKey = (
builtins.readFile (
config.clan.core.settings.directory
+ "/vars/per-machine/${name}/wireguard-keys-${instanceName}/publickey/value"
)
);
allowedIPs = [
"${
builtins.readFile (
config.clan.core.settings.directory
+ "/vars/per-machine/${name}/wireguard-network-${instanceName}/prefix/value"
)
}::/56"
];
endpoint = "${value.settings.endpoint}:${toString value.settings.port}";
persistentKeepalive = 25;
}
) (allPeers // allOtherControllers);
};
};
};
};
# Maps over all machines and produces one result per machine, regardless of role
perMachine =
{ instances, machine, ... }:
{
nixosModule =
{ pkgs, lib, ... }:
let
# Check if this machine has conflicting roles across all instances
machineRoleConflicts = lib.flatten (
lib.mapAttrsToList (
instanceName: instanceInfo:
let
isController =
instanceInfo.roles ? controller && instanceInfo.roles.controller.machines ? ${machine.name};
isPeer = instanceInfo.roles ? peer && instanceInfo.roles.peer.machines ? ${machine.name};
in
lib.optional (isController && isPeer) {
inherit instanceName;
machineName = machine.name;
}
) instances
);
in
{
# Add assertions for role conflicts
assertions = lib.forEach machineRoleConflicts (conflict: {
assertion = false;
message = ''
Machine '${conflict.machineName}' cannot have both 'controller' and 'peer' roles in the wireguard instance '${conflict.instanceName}'.
A machine must be either a controller or a peer, not both.
'';
});
# Generate keys for each instance where this machine participates
clan.core.vars.generators = lib.mapAttrs' (
name: _instanceInfo:
lib.nameValuePair "wireguard-keys-${name}" {
files.publickey.secret = false;
files.privatekey = { };
runtimeInputs = with pkgs; [
wireguard-tools
];
script = ''
wg genkey > $out/privatekey
wg pubkey < $out/privatekey > $out/publickey
'';
}
) instances;
};
};
}

7
clanServices/wireguard/flake-module.nix Normal file
View File

@@ -0,0 +1,7 @@
{ lib, ... }:
let
module = lib.modules.importApply ./default.nix { };
in
{
clan.modules.wireguard = module;
}

135
clanServices/wireguard/ipv6_allocator.py Executable file
View File

@@ -0,0 +1,135 @@
#!/usr/bin/env python3
"""
IPv6 address allocator for WireGuard networks.
Network layout:
- Base network: /40 ULA prefix (fd00::/8 + 32 bits from hash)
- Controllers: Each gets a /56 subnet from the base /40 (256 controllers max)
- Peers: Each gets a /96 subnet from their controller's /56
"""
import hashlib
import ipaddress
import sys
from pathlib import Path
def hash_string(s: str) -> str:
"""Generate SHA256 hash of string."""
return hashlib.sha256(s.encode()).hexdigest()
def generate_ula_prefix(instance_name: str) -> ipaddress.IPv6Network:
"""
Generate a /40 ULA prefix from instance name.
Format: fd{32-bit hash}/40
This gives us fd00:0000:0000::/40 through fdff:ffff:ff00::/40
"""
h = hash_string(instance_name)
# For /40, we need 32 bits after 'fd' (8 hex chars)
# But only the first 32 bits count for the network prefix
# The last 8 bits of the 40-bit prefix must be 0
prefix_bits = int(h[:8], 16)
# Mask to ensure we only use the first 32 bits for /40
# This gives us addresses like fd28:387a::/40
prefix_bits = prefix_bits & 0xFFFFFF00 # Clear last 8 bits
# Format as IPv6 address
prefix = f"fd{prefix_bits:08x}"
prefix_formatted = f"{prefix[:4]}:{prefix[4:8]}::/40"
network = ipaddress.IPv6Network(prefix_formatted)
return network
def generate_controller_subnet(
base_network: ipaddress.IPv6Network, controller_name: str
) -> ipaddress.IPv6Network:
"""
Generate a /56 subnet for a controller from the base /40 network.
We have 16 bits (40 to 56) to allocate controller subnets.
This allows for 65,536 possible controller subnets.
"""
h = hash_string(controller_name)
# Take 16 bits from hash for the controller subnet ID
controller_id = int(h[:4], 16)
# Create the controller subnet by adding the controller ID to the base network
# The controller subnet is at base_prefix:controller_id::/56
base_int = int(base_network.network_address)
controller_subnet_int = base_int | (controller_id << (128 - 56))
controller_subnet = ipaddress.IPv6Network((controller_subnet_int, 56))
return controller_subnet
def generate_peer_suffix(peer_name: str) -> str:
"""
Generate a unique 64-bit host suffix for a peer.
This suffix will be used in all controller subnets to create unique addresses.
Format: :xxxx:xxxx:xxxx:xxxx (64 bits)
"""
h = hash_string(peer_name)
# Take 64 bits (16 hex chars) from hash for the host suffix
suffix_bits = h[:16]
# Format as IPv6 suffix without leading colon
suffix = f"{suffix_bits[0:4]}:{suffix_bits[4:8]}:{suffix_bits[8:12]}:{suffix_bits[12:16]}"
return suffix
def main() -> None:
if len(sys.argv) < 4:
print(
"Usage: ipv6_allocator.py <output_dir> <instance_name> <controller|peer> <machine_name>"
)
sys.exit(1)
output_dir = Path(sys.argv[1])
instance_name = sys.argv[2]
node_type = sys.argv[3]
# Generate base /40 network
base_network = generate_ula_prefix(instance_name)
if node_type == "controller":
if len(sys.argv) < 5:
print("Controller name required")
sys.exit(1)
controller_name = sys.argv[4]
subnet = generate_controller_subnet(base_network, controller_name)
# Extract clean prefix from subnet (e.g. "fd51:19c1:3b:f700::/56" -> "fd51:19c1:3b:f700")
prefix_str = str(subnet).split("/")[0].rstrip(":")
while prefix_str.endswith(":"):
prefix_str = prefix_str.rstrip(":")
# Write file
(output_dir / "prefix").write_text(prefix_str)
elif node_type == "peer":
if len(sys.argv) < 5:
print("Peer name required")
sys.exit(1)
peer_name = sys.argv[4]
# Generate the peer's host suffix
suffix = generate_peer_suffix(peer_name)
# Write file
(output_dir / "suffix").write_text(suffix)
else:
print(f"Unknown node type: {node_type}")
sys.exit(1)
if __name__ == "__main__":
main()