yadunut/clan-core
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

vars: docs

Browse Source 
re-add vars-backend.md

re-add vars-backend.md
This commit is contained in:
Jörg Thalheim
2025-06-27 12:14:26 +02:00
committed by Qubasa
parent e72e100965
commit 79d44f7c30
6 changed files with 857 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
docs/mkdocs.yml
View File

@@ -63,11 +63,19 @@ nav:
- ClanServices: guides/clanServices.md
- Backup & Restore: guides/backups.md
- Disk Encryption: guides/disk-encryption.md
- Age Plugins: guides/age-plugins.md
- Secrets management: guides/secrets.md
- Networking: guides/networking.md
- Zerotier VPN: guides/mesh-vpn.md
- How to disable Secure Boot: guides/secure-boot.md
- Mesh VPN: guides/mesh-vpn.md
- Backup & Restore: guides/backups.md
- Vars:
- Getting Started: guides/vars-backend.md
- Concepts: guides/vars-concepts.md
- Advanced Examples: guides/vars-advanced-examples.md
- Troubleshooting: guides/vars-troubleshooting.md
- Facts Backend: guides/secrets.md
- Adding more machines: guides/more-machines.md
- Target Host: guides/target-host.md
- Inventory:
- Inventory: guides/inventory.md
- Secure Boot: guides/secure-boot.md
- Flake-parts: guides/flake-parts.md
- macOS: guides/macos.md
- Contributing:

6
docs/site/concepts/generators.md
View File

@@ -4,7 +4,7 @@ Defining a linux user's password via the nixos configuration previously required
In this example, we will guide you through automating that interaction using clan `vars`.
For a more general explanation of what clan vars are and how it works, see the intro of the [Reference Documentation for vars](../reference/clan.core/vars.md)
For architectural concepts and design principles, see the [Concepts guide](vars-concepts.md). For the complete API reference, see the [vars module documentation](../reference/clan.core/vars.md).
This guide assumes
- Clan is set up already (see [Getting Started](../guides/getting-started/index.md))
@@ -144,5 +144,9 @@ Updated var root-password/password-hash
## Further Reading
- [Understanding Vars Concepts](vars-concepts.md) - Learn about the architecture and core concepts
- [Advanced Examples](vars-advanced-examples.md) - Complex real-world examples including certificates, SSH keys, and more
- [Troubleshooting Guide](vars-troubleshooting.md) - Common issues and solutions
- [Migration Guide](migrations/migration-facts-vars.md) - Migrate from legacy facts system
- [Reference Documentation for `clan.core.vars` NixOS options](../reference/clan.core/vars.md)
- [Reference Documentation for the `clan vars` CLI command](../reference/cli/vars.md)

289
docs/site/guides/vars-advanced-examples.md Normal file
View File

@@ -0,0 +1,289 @@
# Advanced Vars Examples
This guide demonstrates complex, real-world patterns for the vars system. For basic usage, see the [Getting Started guide](vars-backend.md).
## Certificate Authority with Intermediate Certificates
This example shows how to create a complete certificate authority with root and intermediate certificates using dependencies.
```nix
{
# Generate root CA (not deployed to machines)
clan.core.vars.generators.root-ca = {
files."ca.key" = {
secret = true;
deploy = false; # Keep root key offline
};
files."ca.crt".secret = false;
runtimeInputs = [ pkgs.step-cli ];
script = ''
step certificate create "My Root CA" \
$out/ca.crt $out/ca.key \
--profile root-ca \
--no-password \
--not-after 87600h
'';
};
# Generate intermediate key
clan.core.vars.generators.intermediate-key = {
files."intermediate.key" = {
secret = true;
deploy = true;
};
runtimeInputs = [ pkgs.step-cli ];
script = ''
step crypto keypair \
$out/intermediate.pub \
$out/intermediate.key \
--no-password
'';
};
# Generate intermediate certificate signed by root
clan.core.vars.generators.intermediate-cert = {
files."intermediate.crt".secret = false;
dependencies = [
"root-ca"
"intermediate-key"
];
runtimeInputs = [ pkgs.step-cli ];
script = ''
step certificate create "My Intermediate CA" \
$out/intermediate.crt \
$in/intermediate-key/intermediate.key \
--ca $in/root-ca/ca.crt \
--ca-key $in/root-ca/ca.key \
--profile intermediate-ca \
--not-after 8760h \
--no-password
'';
};
# Use the certificates in services
services.nginx.virtualHosts."example.com" = {
sslCertificate = config.clan.core.vars.generators.intermediate-cert.files."intermediate.crt".value;
sslCertificateKey = config.clan.core.vars.generators.intermediate-key.files."intermediate.key".path;
};
}
```
## Multi-Service Secret Sharing
Generate secrets that multiple services can use:
```nix
{
# Generate database credentials
clan.core.vars.generators.database = {
share = true; # Share across machines
files."password" = { };
files."connection-string" = { };
prompts.dbname = {
description = "Database name";
type = "line";
};
script = ''
# Generate password
tr -dc 'A-Za-z0-9' < /dev/urandom | head -c 32 > $out/password
# Create connection string
echo "postgresql://app:$(cat $out/password)@localhost/$prompts/dbname" \
> $out/connection-string
'';
};
# PostgreSQL uses the password
services.postgresql = {
enable = true;
initialScript = pkgs.writeText "init.sql" ''
CREATE USER app WITH PASSWORD '${
builtins.readFile config.clan.core.vars.generators.database.files."password".path
}';
'';
};
# Application uses the connection string
systemd.services.myapp = {
serviceConfig.EnvironmentFile =
config.clan.core.vars.generators.database.files."connection-string".path;
};
}
```
## SSH Host Keys with Certificates
Generate SSH host keys and sign them with a CA:
```nix
{
# SSH Certificate Authority (shared)
clan.core.vars.generators.ssh-ca = {
share = true;
files."ca" = { secret = true; deploy = false; };
files."ca.pub" = { secret = false; };
runtimeInputs = [ pkgs.openssh ];
script = ''
ssh-keygen -t ed25519 -N "" -f $out/ca
mv $out/ca.pub $out/ca.pub
'';
};
# Host-specific SSH keys
clan.core.vars.generators.ssh-host = {
files."ssh_host_ed25519_key" = {
secret = true;
owner = "root";
group = "root";
mode = "0600";
};
files."ssh_host_ed25519_key.pub" = { secret = false; };
files."ssh_host_ed25519_key-cert.pub" = { secret = false; };
dependencies = [ "ssh-ca" ];
runtimeInputs = [ pkgs.openssh ];
script = ''
# Generate host key
ssh-keygen -t ed25519 -N "" -f $out/ssh_host_ed25519_key
# Sign with CA
ssh-keygen -s $in/ssh-ca/ca \
-I "host:${config.networking.hostName}" \
-h \
-V -5m:+365d \
$out/ssh_host_ed25519_key.pub
'';
};
# Configure SSH to use the generated keys
services.openssh = {
hostKeys = [{
path = config.clan.core.vars.generators.ssh-host.files."ssh_host_ed25519_key".path;
type = "ed25519";
}];
};
}
```
## WireGuard Mesh Network
Create a WireGuard configuration with pre-shared keys:
```nix
{
# Generate WireGuard keys for this host
clan.core.vars.generators.wireguard = {
files."privatekey" = {
secret = true;
owner = "systemd-network";
mode = "0400";
};
files."publickey" = { secret = false; };
files."preshared" = { secret = true; };
runtimeInputs = [ pkgs.wireguard-tools ];
script = ''
# Generate key pair
wg genkey > $out/privatekey
wg pubkey < $out/privatekey > $out/publickey
# Generate pre-shared key
wg genpsk > $out/preshared
'';
};
# Configure WireGuard
networking.wireguard.interfaces.wg0 = {
privateKeyFile = config.clan.core.vars.generators.wireguard.files."privatekey".path;
peers = [{
publicKey = "peer-public-key-here";
presharedKeyFile = config.clan.core.vars.generators.wireguard.files."preshared".path;
allowedIPs = [ "10.0.0.2/32" ];
}];
};
}
```
## Conditional Generation Based on Machine Role
Generate different secrets based on machine configuration:
```nix
{
clan.core.vars.generators = lib.mkMerge [
# All machines get basic auth
{
basic-auth = {
files."htpasswd" = { };
prompts.username = {
description = "Username for basic auth";
type = "line";
};
prompts.password = {
description = "Password for basic auth";
type = "hidden";
};
runtimeInputs = [ pkgs.apacheHttpd ];
script = ''
htpasswd -nbB "$prompts/username" "$prompts/password" > $out/htpasswd
'';
};
}
# Only servers get API tokens
(lib.mkIf config.services.myapi.enable {
api-tokens = {
files."admin-token" = { };
files."readonly-token" = { };
runtimeInputs = [ pkgs.openssl ];
script = ''
openssl rand -hex 32 > $out/admin-token
openssl rand -hex 16 > $out/readonly-token
'';
};
})
];
}
```
## Backup Encryption Keys
Generate and manage backup encryption keys:
```nix
{
clan.core.vars.generators.backup = {
share = true; # Same key for all backup sources
files."encryption.key" = {
secret = true;
deploy = true;
};
files."encryption.pub" = { secret = false; };
runtimeInputs = [ pkgs.age ];
script = ''
# Generate age key pair
age-keygen -o $out/encryption.key 2>/dev/null
# Extract public key
grep "public key:" $out/encryption.key | cut -d: -f2 | tr -d ' ' \
> $out/encryption.pub
'';
};
# Use in backup service
services.borgbackup.jobs.system = {
encryption = {
mode = "repokey-blake2";
passCommand = "cat ${config.clan.core.vars.generators.backup.files."encryption.key".path}";
};
};
}
```
## Tips and Best Practices
1. **Use dependencies** to build complex multi-stage generations
2. **Share generators** when the same secret is needed across machines
3. **Set appropriate permissions** for service-specific secrets
4. **Use prompts** for user-specific values that shouldn't be generated
5. **Combine secret and non-secret files** in the same generator when they're related
6. **Use conditional generation** with `lib.mkIf` for role-specific secrets

155
docs/site/guides/vars-backend.md Normal file
View File

@@ -0,0 +1,155 @@
!!! Note
This guide demonstrates the vars system for managing secrets and generated files
Defining a linux user's password via the nixos configuration previously required running `mkpasswd ...` and then copying the hash back into the nix configuration.
In this example, we will guide you through automating that interaction using clan `vars`.
For architectural concepts and design principles, see the [Concepts guide](vars-concepts.md). For the complete API reference, see the [vars module documentation](../reference/clan.core/vars.md).
This guide assumes
- Clan is set up already (see [Getting Started](../guides/getting-started/index.md))
- a machine has been added to the clan (see [Adding Machines](./more-machines.md))
This section will walk you through the following steps:
1. declare a `generator` in the machine's nixos configuration
2. inspect the status via the Clan CLI
3. generate the vars
4. observe the changes
5. update the machine
6. share the root password between machines
7. change the password
## Declare the generator
In this example, a `vars` `generator` is used to:
- prompt the user for the password
- run the required `mkpasswd` command to generate the hash
- store the hash in a file
- expose the file path to the nixos configuration
Create a new nix file `root-password.nix` with the following content and import it into your `configuration.nix`
```nix
{config, pkgs, ...}: {
clan.core.vars.generators.root-password = {
# prompt the user for a password
# (`password-input` being an arbitrary name)
prompts.password-input.description = "the root user's password";
prompts.password-input.type = "hidden";
# don't store the prompted password itself
prompts.password-input.persist = false;
# define an output file for storing the hash
files.password-hash.secret = false;
# define the logic for generating the hash
script = ''
cat $prompts/password-input | mkpasswd -m sha-512 > $out/password-hash
'';
# the tools required by the script
runtimeInputs = [ pkgs.mkpasswd ];
};
# ensure users are immutable (otherwise the following config might be ignored)
users.mutableUsers = false;
# set the root password to the file containing the hash
users.users.root.hashedPasswordFile =
# clan will make sure, this path exists
config.clan.core.vars.generators.root-password.files.password-hash.path;
}
```
## Inspect the status
Executing `clan vars list`, you should see the following:
```shellSession
$ clan vars list my_machine
root-password/password-hash: <not set>
```
...indicating that the value `password-hash` for the generator `root-password` is not set yet.
## Generate the values
This step is not strictly necessary, as deploying the machine via `clan machines update` would trigger the generator as well.
To run the generator, execute `clan vars generate` for your machine
```shellSession
$ clan vars generate my_machine
Enter the value for root-password/password-input (hidden):
```
After entering the value, the updated status is reported:
```shellSession
Updated var root-password/password-hash
old: <not set>
new: $6$RMats/YMeypFtcYX$DUi...
```
## Observe the changes
With the last step, a new file was created in your repository:
`vars/per-machine/my-machine/root-password/password-hash/value`
If the repository is a git repository, a commit was created automatically:
```shellSession
$ git log -n1
commit ... (HEAD -> master)
Author: ...
Date: ...
Update vars via generator root-password for machine grmpf-nix
```
## Update the machine
```shell
clan machines update my_machine
```
## Share root password between machines
If we just imported the `root-password.nix` from above into more machines, clan would ask for a new password for each additional machine.
If the root password instead should only be entered once and shared across all machines, the generator defined above needs to be declared as `shared`, by adding `share = true` to it:
```nix
{config, pkgs, ...}: {
clan.vars.generators.root-password = {
share = true;
# ...
}
}
```
Importing that shared generator into each machine, will ensure that the password is only asked once the first machine gets updated and then re-used for all subsequent machines.
## Change the root password
Changing the password can be done via this command.
Replace `my-machine` with your machine.
If the password is shared, just pick any machine that has the generator declared.
```shellSession
$ clan vars generate my-machine --generator root-password --regenerate
...
Enter the value for root-password/password-input (hidden):
Input received. Processing...
...
Updated var root-password/password-hash
old: $6$tb27m6EOdff.X9TM$19N...
new: $6$OyoQtDVzeemgh8EQ$zRK...
```
## Further Reading
- [Understanding Vars Concepts](vars-concepts.md) - Learn about the architecture and core concepts
- [Advanced Examples](vars-advanced-examples.md) - Complex real-world examples including certificates, SSH keys, and more
- [Troubleshooting Guide](vars-troubleshooting.md) - Common issues and solutions
- [Migration Guide](migrations/migration-facts-vars.md) - Migrate from legacy facts system
- [Reference Documentation for `clan.core.vars` NixOS options](../reference/clan.core/vars.md)
- [Reference Documentation for the `clan vars` CLI command](../reference/cli/vars.md)

129
docs/site/guides/vars-concepts.md Normal file
View File

@@ -0,0 +1,129 @@
# Understanding Clan Vars - Concepts & Architecture
This guide explains the architecture and design principles behind the vars system. For a hands-on tutorial, see the [Getting Started guide](vars-backend.md).
## Architecture Overview
The vars system provides a declarative, reproducible way to manage generated files (especially secrets) in NixOS configurations.
## Data Flow
```mermaid
graph LR
A[Generator Script] --> B[Output Files]
C[User Prompts] --> A
D[Dependencies] --> A
B --> E[Secret Storage<br/>sops/password-store]
B --> F[Nix Store<br/>public files]
E --> G[Machine Deployment]
F --> G
```
## Key Design Principles
### 1. Declarative Generation
Unlike imperative secret management, vars are declared in your NixOS configuration and generated deterministically. This ensures reproducibility across deployments.
### 2. Separation of Concerns
- **Generation logic**: Defined in generator scripts
- **Storage**: Handled by pluggable backends (sops, password-store, etc.)
- **Deployment**: Managed by NixOS activation scripts
- **Access control**: Enforced through file permissions and ownership
### 3. Composability Through Dependencies
Generators can depend on outputs from other generators, enabling complex workflows:
```nix
# Dependencies create a directed acyclic graph (DAG)
A → B → C
↓
D
```
This allows building sophisticated systems like certificate authorities where intermediate certificates depend on root certificates.
### 4. Type Safety
The vars system distinguishes between:
- **Secret files**: Only accessible via `.path`, deployed to `/run/secrets/`
- **Public files**: Accessible via `.value`, stored in nix store
This prevents accidental exposure of secrets in the nix store.
## Storage Backend Architecture
The vars system uses pluggable storage backends:
- **sops** (default): Integrates with clan's existing sops encryption
- **password-store**: For users already using pass
Each backend handles encryption/decryption transparently, allowing the same generator definitions to work across different security models.
## Timing and Lifecycle
### Generation Phases
1. **Pre-deployment**: `clan vars generate` creates vars before deployment
2. **During deployment**: Missing vars are generated automatically
3. **Regeneration**: Explicit regeneration with `--regenerate` flag
### The `neededFor` Option
Control when vars are available during system activation:
```nix
files."early-secret" = {
secret = true;
neededFor = [ "users" "groups" ]; # Available early in activation
};
```
## Advanced Patterns
### Multi-Machine Coordination
The `share` option enables cross-machine secret sharing:
```mermaid
graph LR
A[Shared Generator] --> B[Machine 1]
A --> C[Machine 2]
A --> D[Machine 3]
```
This is useful for:
- Shared certificate authorities
- Mesh VPN pre-shared keys
- Cluster join tokens
### Generator Composition
Complex systems can be built by composing simple generators:
```
root-ca → intermediate-ca → service-cert
↓
ocsp-responder
```
Each generator focuses on one task, making the system modular and testable.
## Key Advantages
Compared to manual secret management, vars provides:
- **Declarative configuration**: Define once, generate consistently
- **Dependency management**: Build complex systems with generator dependencies
- **Type safety**: Separate handling of secret and public files
- **User prompts**: Gather input when needed
- **Easy regeneration**: Update secrets with a single command
## Next Steps
- [Practical Tutorial](vars-backend.md) - Step-by-step guide
- [Advanced Examples](vars-advanced-examples.md) - Real-world patterns
- [Troubleshooting](vars-troubleshooting.md) - Common issues
- [Reference](../reference/clan.core/vars.md) - Complete API

266
docs/site/guides/vars-troubleshooting.md Normal file
View File

@@ -0,0 +1,266 @@
# Troubleshooting Vars
Quick reference for diagnosing and fixing vars issues. For basic usage, see the [Getting Started guide](vars-backend.md).
## Common Issues
### Generator Script Fails
**Symptom**: Error during `clan vars generate` or deployment
**Possible causes and solutions**:
1. **Missing runtime inputs**
```nix
# Wrong - missing required tool
runtimeInputs = [ ];
script = ''
openssl rand -hex 32 > $out/secret # openssl not found!
'';
# Correct
runtimeInputs = [ pkgs.openssl ];
```
2. **Wrong output path**
```nix
# Wrong - must use $out
script = ''
echo "secret" > ./myfile
'';
# Correct
script = ''
echo "secret" > $out/myfile
'';
```
3. **Missing declared files**
```nix
files."config" = { };
files."key" = { };
script = ''
# Wrong - only generates one file
echo "data" > $out/config
'';
# Correct - must generate all declared files
script = ''
echo "data" > $out/config
echo "key" > $out/key
'';
```
### Cannot Access Generated Files
**Symptom**: "attribute 'value' missing" or file not found
**Solutions**:
1. **Secret files don't have `.value`**
```nix
# Wrong - secret files can't use .value
files."secret" = { secret = true; };
# ...
environment.etc."app.conf".text =
config.clan.core.vars.generators.app.files."secret".value;
# Correct - use .path for secrets
environment.etc."app.conf".source =
config.clan.core.vars.generators.app.files."secret".path;
```
2. **Public files should use `.value`**
```nix
# Better for non-secrets
files."cert.pem" = { secret = false; };
# ...
sslCertificate =
config.clan.core.vars.generators.ca.files."cert.pem".value;
```
### Dependencies Not Available
**Symptom**: "No such file or directory" when accessing `$in/...`
**Solution**: Declare dependencies correctly
```nix
clan.core.vars.generators.child = {
# Wrong - missing dependency
script = ''
cat $in/parent/file > $out/newfile
'';
# Correct
dependencies = [ "parent" ];
script = ''
cat $in/parent/file > $out/newfile
'';
};
```
### Permission Denied
**Symptom**: Service cannot read generated secret file
**Solution**: Set correct ownership and permissions
```nix
files."service.key" = {
secret = true;
owner = "myservice"; # Match service user
group = "myservice";
mode = "0400"; # Read-only for owner
};
```
### Vars Not Regenerating
**Symptom**: Changes to generator script don't trigger regeneration
**Solution**: Use `--regenerate` flag
```bash
clan vars generate my-machine --generator my-generator --regenerate
```
### Prompts Not Working
**Symptom**: Script fails with "No such file or directory" for prompts
**Solution**: Access prompts correctly
```nix
# Wrong
script = ''
echo $password > $out/file
'';
# Correct
prompts.password.type = "hidden";
script = ''
cat $prompts/password > $out/file
'';
```
## Debugging Techniques
### 1. Check Generator Status
See what vars are set:
```bash
clan vars list my-machine
```
### 2. Inspect Generated Files
For shared vars:
```bash
ls -la vars/shared/my-generator/
```
For per-machine vars:
```bash
ls -la vars/per-machine/my-machine/my-generator/
```
### 3. Test Generators Locally
Create a test script to debug:
```nix
# test-generator.nix
{ pkgs ? import <nixpkgs> {} }:
pkgs.stdenv.mkDerivation {
name = "test-generator";
buildInputs = [ pkgs.openssl ]; # Your runtime inputs
buildCommand = ''
# Your generator script here
mkdir -p $out
openssl rand -hex 32 > $out/secret
ls -la $out/
'';
}
```
Run with:
```bash
nix-build test-generator.nix
```
### 4. Enable Debug Logging
Set debug mode:
```bash
clan --debug vars generate my-machine
```
### 5. Check File Permissions
Verify generated secret permissions:
```bash
# On the target machine
ls -la /run/secrets/
```
## Recovery Procedures
### Regenerate All Vars
If vars are corrupted or need refresh:
```bash
# Regenerate all for a machine
clan vars generate my-machine --regenerate
# Regenerate specific generator
clan vars generate my-machine --generator my-generator --regenerate
```
### Manual Secret Injection
For recovery or testing:
```bash
# Set a var manually (bypass generator)
echo "temporary-secret" | clan vars set my-machine my-generator/my-file
```
### Restore from Backup
Vars are stored in the repository:
```bash
# Restore previous version
git checkout HEAD~1 -- vars/
# Check and regenerate if needed
clan vars list my-machine
```
## Storage Backend Issues
### SOPS Decryption Fails
**Symptom**: "Failed to decrypt" or permission errors
**Solution**: Ensure your user/machine has the correct age keys configured. Clan manages encryption keys automatically based on the configured users and machines in your flake.
Check that:
1. Your machine is properly configured in the flake
2. Your user has access to the machine's secrets
3. The age key is available in the expected location
### Password Store Issues
**Symptom**: "pass: store not initialized"
**Solution**: Initialize password store:
```bash
export PASSWORD_STORE_DIR=/path/to/clan/vars
pass init your-gpg-key
```
## Getting Help
If these solutions don't resolve your issue:
1. Check the [clan-core issue tracker](https://git.clan.lol/clan/clan-core/issues)
2. Ask in the Clan community channels
3. Provide:
- The generator configuration
- The exact error message
- Output of `clan --debug vars generate`