docs: extensive backup documentation
This commit is contained in:
Qubasa
@@ -63,7 +63,6 @@ nav:
|
||||
- Inventory:
|
||||
- Introduction to Inventory: guides/inventory/inventory.md
|
||||
- File Autoincludes: guides/inventory/autoincludes.md
|
||||
|
||||
- Clan Services:
|
||||
- Inventory Guide: guides/inventory/clanServices.md
|
||||
- Author Your Own Service: guides/services/community.md
|
||||
@@ -76,14 +75,17 @@ nav:
|
||||
- Sops Backend:
|
||||
- Yubikeys & Age Plugins: guides/vars/sops/age-plugins.md
|
||||
- Managing Users (OLD): guides/secrets.md
|
||||
- Backups:
|
||||
- Introduction to Backups: guides/backups/backup-intro.md
|
||||
- Minimal Example: guides/backups/minimal-example.md
|
||||
- Digging Deeper: guides/backups/digging-deeper.md
|
||||
- Advanced Example: guides/backups/advanced-example.md
|
||||
- Networking:
|
||||
- Introduction to Networking: guides/networking/networking.md
|
||||
- Zerotier VPN: guides/networking/mesh-vpn.md
|
||||
- Nixpkgs Flake Input: guides/nixpkgs-flake-input/index.md
|
||||
- Disko Templates:
|
||||
- Community Disko Templates: guides/disko-templates/community.md
|
||||
- Backups:
|
||||
- Introduction to Backups: guides/backups.md
|
||||
- Flake-parts: guides/flake-parts.md
|
||||
- NixOS Rebuild: guides/nixos-rebuild.md
|
||||
- macOS:
|
||||
|
||||
@@ -1,195 +0,0 @@
|
||||
This guide explains how to set up and manage
|
||||
[BorgBackup](https://borgbackup.readthedocs.io/) for secure, efficient backups
|
||||
in a clan network. BorgBackup provides:
|
||||
|
||||
- Space efficient storage of backups with deduplication
|
||||
- Secure, authenticated encryption
|
||||
- Compression: lz4, zstd, zlib, lzma or none
|
||||
- Mountable backups with FUSE
|
||||
- Easy installation on multiple platforms: Linux, macOS, BSD, …
|
||||
- Free software (BSD license).
|
||||
- Backed by a large and active open-source community.
|
||||
|
||||
## Borgbackup Example
|
||||
|
||||
```nix
|
||||
inventory.instances = {
|
||||
borgbackup = {
|
||||
module = {
|
||||
name = "borgbackup";
|
||||
input = "clan-core";
|
||||
};
|
||||
roles.client.machines."jon".settings = {
|
||||
destinations."storagebox" = {
|
||||
repo = "username@$hostname:/./borgbackup";
|
||||
rsh = ''ssh -oPort=23 -i /run/secrets/vars/borgbackup/borgbackup.ssh'';
|
||||
};
|
||||
};
|
||||
roles.server.machines = { };
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
The input should be named according to your flake input. Jon is configured as a
|
||||
client machine with a destination pointing to a Hetzner Storage Box.
|
||||
|
||||
To see a list of all possible options go to [borgbackup clan service](../reference/clanServices/borgbackup.md)
|
||||
|
||||
## Roles
|
||||
|
||||
A Clan Service can have multiple roles, each role applies different nix config to the machine.
|
||||
|
||||
### 1. Client
|
||||
|
||||
Clients are machines that create and send backups to various destinations. Each
|
||||
client can have multiple backup destinations configured.
|
||||
|
||||
### 2. Server
|
||||
|
||||
Servers act as backup repositories, receiving and storing backups from client
|
||||
machines. They can be dedicated backup servers within your clan network.
|
||||
|
||||
## Backup destinations
|
||||
|
||||
This service allows you to perform backups to multiple `destinations`.
|
||||
Destinations can be:
|
||||
|
||||
- **Local**: Local disk storage
|
||||
- **Server**: Your own borgbackup server (using the `server` role)
|
||||
- **Third-party services**: Such as Hetzner's Storage Box
|
||||
|
||||
## State management
|
||||
|
||||
Backups are based on [states](../reference/clan.core/state.md). A state
|
||||
defines which files should be backed up and how these files are obtained through
|
||||
pre/post backup and restore scripts.
|
||||
|
||||
Here's an example for a user application `linkding`:
|
||||
|
||||
In this example:
|
||||
|
||||
- `/data/podman/linkding` is the application's data directory
|
||||
- `/var/backup/linkding` is the staging directory where data is copied for
|
||||
backup
|
||||
|
||||
```nix
|
||||
clan.core.state.linkding = {
|
||||
folders = [ "/var/backup/linkding" ];
|
||||
|
||||
preBackupScript = ''
|
||||
export PATH=${
|
||||
lib.makeBinPath [
|
||||
config.systemd.package
|
||||
pkgs.coreutils
|
||||
pkgs.rsync
|
||||
]
|
||||
}
|
||||
|
||||
service_status=$(systemctl is-active podman-linkding)
|
||||
|
||||
if [ "$service_status" = "active" ]; then
|
||||
systemctl stop podman-linkding
|
||||
rsync -avH --delete --numeric-ids "/data/podman/linkding/" /var/backup/linkding/
|
||||
systemctl start podman-linkding
|
||||
fi
|
||||
'';
|
||||
|
||||
postRestoreScript = ''
|
||||
export PATH=${
|
||||
lib.makeBinPath [
|
||||
config.systemd.package
|
||||
pkgs.coreutils
|
||||
pkgs.rsync
|
||||
]
|
||||
}
|
||||
|
||||
service_status="$(systemctl is-active podman-linkding)"
|
||||
|
||||
if [ "$service_status" = "active" ]; then
|
||||
systemctl stop podman-linkding
|
||||
|
||||
# Backup locally current linkding data
|
||||
cp -rp "/data/podman/linkding" "/data/podman/linkding.bak"
|
||||
|
||||
# Restore from borgbackup
|
||||
rsync -avH --delete --numeric-ids /var/backup/linkding/ "/data/podman/linkding/"
|
||||
|
||||
systemctl start podman-linkding
|
||||
fi
|
||||
'';
|
||||
};
|
||||
```
|
||||
|
||||
## Managing backups
|
||||
|
||||
In this section we go over how to manage your collection of backups with the clan command.
|
||||
|
||||
### Listing states
|
||||
|
||||
To see which files (`states`) will be backed up on a specific machine, use:
|
||||
|
||||
```bash
|
||||
clan state list jon
|
||||
```
|
||||
|
||||
This will show all configured states for the machine `jon`, for example:
|
||||
|
||||
```text
|
||||
· service: linkding
|
||||
folders:
|
||||
- /var/backup/linkding
|
||||
preBackupCommand: pre-backup-linkding
|
||||
postRestoreCommand: post-restore-linkding
|
||||
|
||||
· service: zerotier
|
||||
folders:
|
||||
- /var/lib/zerotier-one
|
||||
```
|
||||
|
||||
### Creating backups
|
||||
|
||||
To create a backup of a machine (e.g., `jon`), run:
|
||||
|
||||
```bash
|
||||
clan backups create jon
|
||||
```
|
||||
|
||||
This will backup all configured states (`zerotier` and `linkding` in this
|
||||
example) from the machine `jon`.
|
||||
|
||||
### Listing available backups
|
||||
|
||||
To see all available backups, use:
|
||||
|
||||
```bash
|
||||
clan backups list
|
||||
```
|
||||
|
||||
This will display all backups with their timestamps:
|
||||
|
||||
```text
|
||||
storagebox::username@username.your-storagebox.de:/./borgbackup::jon-jon-2025-07-22T19:40:10
|
||||
storagebox::username@username.your-storagebox.de:/./borgbackup::jon-jon-2025-07-23T01:00:00
|
||||
storagebox::username@username.your-storagebox.de:/./borgbackup::jon-storagebox-2025-07-24T01:00:00
|
||||
storagebox::username@username.your-storagebox.de:/./borgbackup::jon-storagebox-2025-07-24T06:02:35
|
||||
```
|
||||
|
||||
### Restoring backups
|
||||
|
||||
For restoring a backup you have two options.
|
||||
|
||||
#### Full restoration
|
||||
|
||||
To restore all services from a backup:
|
||||
|
||||
```bash
|
||||
clan backups restore jon borgbackup storagebox::u444061@u444061.your-storagebox.de:/./borgbackup::jon-storagebox-2025-07-24T06:02:35
|
||||
```
|
||||
|
||||
#### Partial restoration
|
||||
|
||||
To restore only a specific service (e.g., `linkding`):
|
||||
|
||||
```bash
|
||||
clan backups restore --service linkding jon borgbackup storagebox::u444061@u444061.your-storagebox.de:/./borgbackup::jon-storagebox-2025-07-24T06:02:35
|
||||
```
|
||||
71
docs/site/guides/backups/advanced-example.md
Normal file
71
docs/site/guides/backups/advanced-example.md
Normal file
@@ -0,0 +1,71 @@
|
||||
This guide explains how to set up a [Hetzner Storage Box](https://docs.hetzner.com/storage/storage-box/general) as a backup destination instead of using an internal Clan backup server. Follow the steps below to configure and verify the setup.
|
||||
|
||||
### Step 1: Create a Hetzner Storage Box
|
||||
|
||||
Begin by [creating a Hetzner Storage Box account](https://docs.hetzner.com/storage/storage-box/getting-started/creating-a-storage-box).
|
||||
|
||||
### Step 2: Create a Sub-Account
|
||||
|
||||
Set up a sub-account for your `jon` machine. Save the SSH password for this account in your password manager for future reference.
|
||||
|
||||
### Step 3: Configure BorgBackup in `clan.nix`
|
||||
|
||||
Add the BorgBackup service to your `clan.nix` configuration. In this example, the `jon` machine will back up to `user-sub1@user-sub1.your-storagebox.de` in the `borgbackup` folder:
|
||||
|
||||
```nix hl_lines="9"
|
||||
inventory.instances = {
|
||||
borgbackup = {
|
||||
module = {
|
||||
name = "borgbackup";
|
||||
input = "clan-core";
|
||||
};
|
||||
roles.client.machines."jon".settings = {
|
||||
destinations."storagebox" = {
|
||||
repo = "user-sub1@user-sub1.your-storagebox.de:/./borgbackup";
|
||||
rsh = ''ssh -p 23 -oStrictHostKeyChecking=accept-new -i /run/secrets/vars/borgbackup/borgbackup.ssh'';
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
### Step 4: Generate SSH Keys
|
||||
|
||||
Run the following command to generate the SSH private keys:
|
||||
|
||||
```bash
|
||||
clan vars generate
|
||||
```
|
||||
|
||||
### Step 5: Add the Public Key to the Sub-Account
|
||||
|
||||
Add the generated SSH public key to the `user-sub1` account by running:
|
||||
|
||||
```bash
|
||||
clan vars get jon borgbackup/borgbackup.ssh.pub | ssh -p23 user-sub1@user-sub1.your-storagebox.de install-ssh-key
|
||||
```
|
||||
|
||||
### Step 6: Deploy the Configuration
|
||||
|
||||
Apply the changes to your Clan setup by executing:
|
||||
|
||||
```bash
|
||||
clan machines update
|
||||
```
|
||||
|
||||
### Step 7: Verify the Setup
|
||||
|
||||
Check if the configuration works by starting the BorgBackup service on the `jon` machine:
|
||||
|
||||
```bash
|
||||
systemctl start borgbackup-job-storagebox.service &
|
||||
```
|
||||
|
||||
Then, inspect the service logs to ensure everything is functioning correctly:
|
||||
|
||||
```bash
|
||||
journalctl -u borgbackup-job-storagebox.service
|
||||
```
|
||||
|
||||
|
||||
|
||||
89
docs/site/guides/backups/backup-intro.md
Normal file
89
docs/site/guides/backups/backup-intro.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# Introduction to Clan Backups
|
||||
|
||||
This guide explains how to use the Clan backup and state management interface to configure, manage, and restore backups for your services and machines. By the end of this guide, you will understand how to define backup states, manage backups, and restore data.
|
||||
|
||||
## State Management
|
||||
|
||||
Clan backups are based on the concept of [states](../../reference/clan.core/state.md). A state is a Nix attribute set, defined as `clan.core.state.<name> = {};`, which specifies the files or directories to back up.
|
||||
|
||||
For example, if you have a clan service called `linkding`, you can define the folders to back up as follows:
|
||||
|
||||
```nix hl_lines="2"
|
||||
clan.core.state.linkding = {
|
||||
folders = [ "/var/backup/linkding" ];
|
||||
};
|
||||
```
|
||||
|
||||
In this example:
|
||||
|
||||
- `/var/backup/linkding` is the staging directory where data is prepared for backup.
|
||||
|
||||
This simple configuration ensures that all critical data for the `linkding` service is included in the backup process.
|
||||
|
||||
|
||||
## Custom Pre and Post Backup Hooks
|
||||
|
||||
The state interface allows you to run custom scripts before creating a backup and after restoring one. These scripts are defined using the `preBackupScript` and `postRestoreScript` options. This can be useful for tasks like stopping services, syncing data, or performing cleanup operations.
|
||||
|
||||
### Example: Pre and Post Backup Scripts for the `linkding` Service
|
||||
|
||||
In the following example, we configure the `linkding` service to:
|
||||
|
||||
1. Stop the service before backing up its data.
|
||||
2. Sync the data to a staging directory.
|
||||
3. Restore the data and restart the service after restoration.
|
||||
|
||||
```nix hl_lines="5 26"
|
||||
clan.core.state.linkding = {
|
||||
folders = [ "/var/backup/linkding" ];
|
||||
|
||||
# Script to run before creating a backup
|
||||
preBackupScript = ''
|
||||
export PATH=${
|
||||
lib.makeBinPath [
|
||||
config.systemd.package
|
||||
pkgs.coreutils
|
||||
pkgs.rsync
|
||||
]
|
||||
}
|
||||
|
||||
# Check if the service is running
|
||||
service_status=$(systemctl is-active podman-linkding)
|
||||
|
||||
if [ "$service_status" = "active" ]; then
|
||||
# Stop the service and sync data to the backup directory
|
||||
systemctl stop podman-linkding
|
||||
rsync -avH --delete --numeric-ids "/data/podman/linkding/" /var/backup/linkding/
|
||||
systemctl start podman-linkding
|
||||
fi
|
||||
'';
|
||||
|
||||
# Script to run after restoring a backup
|
||||
postRestoreScript = ''
|
||||
export PATH=${
|
||||
lib.makeBinPath [
|
||||
config.systemd.package
|
||||
pkgs.coreutils
|
||||
pkgs.rsync
|
||||
]
|
||||
}
|
||||
|
||||
# Check if the service is running
|
||||
service_status="$(systemctl is-active podman-linkding)"
|
||||
|
||||
if [ "$service_status" = "active" ]; then
|
||||
# Stop the service
|
||||
systemctl stop podman-linkding
|
||||
|
||||
# Backup current data locally
|
||||
cp -rp "/data/podman/linkding" "/data/podman/linkding.bak"
|
||||
|
||||
# Restore data from the backup directory
|
||||
rsync -avH --delete --numeric-ids /var/backup/linkding/ "/data/podman/linkding/"
|
||||
|
||||
# Restart the service
|
||||
systemctl start podman-linkding
|
||||
fi
|
||||
'';
|
||||
};
|
||||
```
|
||||
75
docs/site/guides/backups/digging-deeper.md
Normal file
75
docs/site/guides/backups/digging-deeper.md
Normal file
@@ -0,0 +1,75 @@
|
||||
|
||||
|
||||
In this section we go over how to manage your collection of backups with the clan command.
|
||||
|
||||
### Listing states
|
||||
|
||||
To see which files (`states`) will be backed up on a specific machine, use:
|
||||
|
||||
```bash
|
||||
clan state list jon
|
||||
```
|
||||
|
||||
This will show all configured states for the machine `jon`, for example:
|
||||
|
||||
```text
|
||||
· service: linkding
|
||||
folders:
|
||||
- /var/backup/linkding
|
||||
preBackupCommand: pre-backup-linkding
|
||||
postRestoreCommand: post-restore-linkding
|
||||
|
||||
· service: zerotier
|
||||
folders:
|
||||
- /var/lib/zerotier-one
|
||||
```
|
||||
|
||||
### Creating backups
|
||||
|
||||
To create a backup of a machine (e.g., `jon`), run:
|
||||
|
||||
```bash
|
||||
clan backups create jon
|
||||
```
|
||||
|
||||
This will backup all configured states (`zerotier` and `linkding` in this
|
||||
example) from the machine `jon`.
|
||||
|
||||
### Listing available backups
|
||||
|
||||
To see all available backups, use:
|
||||
|
||||
```bash
|
||||
clan backups list
|
||||
```
|
||||
|
||||
This will display all backups with their timestamps:
|
||||
|
||||
```text
|
||||
storagebox::username@username.your-storagebox.de:/./borgbackup::jon-jon-2025-07-22T19:40:10
|
||||
storagebox::username@username.your-storagebox.de:/./borgbackup::jon-jon-2025-07-23T01:00:00
|
||||
storagebox::username@username.your-storagebox.de:/./borgbackup::jon-storagebox-2025-07-24T01:00:00
|
||||
storagebox::username@username.your-storagebox.de:/./borgbackup::jon-storagebox-2025-07-24T06:02:35
|
||||
```
|
||||
|
||||
### Restoring backups
|
||||
|
||||
For restoring a backup you have two options.
|
||||
|
||||
#### Full restoration
|
||||
|
||||
To restore all services from a backup:
|
||||
|
||||
```bash
|
||||
clan backups restore jon borgbackup storagebox::u444061@u444061.your-storagebox.de:/./borgbackup::jon-storagebox-2025-07-24T06:02:35
|
||||
```
|
||||
|
||||
#### Partial restoration
|
||||
|
||||
To restore only a specific service (e.g., `linkding`):
|
||||
|
||||
```bash
|
||||
clan backups restore --service linkding jon borgbackup storagebox::u444061@u444061.your-storagebox.de:/./borgbackup::jon-storagebox-2025-07-24T06:02:35
|
||||
```
|
||||
|
||||
|
||||
63
docs/site/guides/backups/minimal-example.md
Normal file
63
docs/site/guides/backups/minimal-example.md
Normal file
@@ -0,0 +1,63 @@
|
||||
In this guide we will explain how to install a simple peer-to-peer backup system through the inventory. Such that machines will backup it's state to other machines in the clan, ensuring redundancy and data safety.
|
||||
|
||||
|
||||
### What is BorgBackup?
|
||||
|
||||
BorgBackup is a powerful and efficient backup solution designed for secure and space-efficient backups. It offers features such as:
|
||||
|
||||
- **Deduplication**: Saves storage space by avoiding duplicate data.
|
||||
- **Encryption**: Ensures backups are secure and authenticated.
|
||||
- **Compression**: Supports multiple compression algorithms like lz4, zstd, zlib, and more.
|
||||
- **FUSE Mounting**: Allows backups to be mounted as a file system.
|
||||
- **Cross-Platform**: Works on Linux, macOS, BSD, and more.
|
||||
- **Open Source**: Licensed under BSD and supported by an active community.
|
||||
|
||||
|
||||
While this guide uses BorgBackup, you can also use other backup services supported by Clan, depending on your requirements.
|
||||
|
||||
|
||||
### Example Setup
|
||||
|
||||
In this example, we configure a backup system with three machines: `bob`, `jon`, and `alice`. The `bob` and `jon` machines will periodically back up their state folders to `alice`. The backups are encrypted for security.
|
||||
|
||||
|
||||
|
||||
```nix
|
||||
inventory.instances = {
|
||||
borgbackup = {
|
||||
module = {
|
||||
name = "borgbackup";
|
||||
input = "clan-core";
|
||||
};
|
||||
roles.client.machines = {
|
||||
"bob" = { };
|
||||
"jon" = { };
|
||||
};
|
||||
roles.server.machines = {
|
||||
"alice" = { };
|
||||
};
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
|
||||
## Roles
|
||||
|
||||
In a Clan Service, roles define how machines participate in the backup system. Each role applies specific Nix configurations to the machine, enabling flexibility and scalability in your backup setup.
|
||||
|
||||
- **Client**: These machines create backups and send them to designated destinations. Clients can be configured to back up to multiple destinations, ensuring redundancy and reliability.
|
||||
|
||||
- **Server**: These machines act as repositories, receiving and securely storing backups from client machines. Servers can be dedicated backup nodes within your clan network, providing centralized storage for all backups.
|
||||
|
||||
|
||||
## Backup destinations
|
||||
|
||||
This service allows you to perform backups to multiple `destinations`.
|
||||
Destinations can be:
|
||||
|
||||
- **Local**: Local disk storage
|
||||
- **Server**: Your own borgbackup server (using the `server` role)
|
||||
- **Third-party services**: Such as Hetzner's Storage Box
|
||||
|
||||
|
||||
However, if BorgBackup does not meet your needs, you can implement your own backup clan service.
|
||||
@@ -74,7 +74,7 @@ hide:
|
||||
<input type="checkbox" id="clan-readmore" class="clamp-toggle" />
|
||||
<div class="clamp-content">
|
||||
<p><a href="https://clan.lol/">Clan</a> is a peer-to-peer computer management framework that empowers you to selfhost in a reliable and scalable way</strong>.</p>
|
||||
<p>Built on NixOS, Clan provides a declarative interface for managing machines</strong> with automated <a href="./guides/secrets.md">secret management</a>, easy <a href="./guides/mesh-vpn.md">mesh VPN connectivity</a>, and <a href="./guides/backups.md">automated backups</a>.</p>
|
||||
<p>Built on NixOS, Clan provides a declarative interface for managing machines</strong> with automated <a href="./guides/secrets.md">secret management</a>, easy <a href="./guides/mesh-vpn.md">mesh VPN connectivity</a>, and <a href="./guides/backups/backup-intro.md">automated backups</a>.</p>
|
||||
<p>Whether you're running a homelab or maintaining critical computing infrastructure, Clan will help reduce maintenance burden</strong> by allowing a git repository to define your whole network</strong> of computers.</p>
|
||||
<p>In combination with <a href="https://github.com/Mic92/sops-nix">sops-nix</a>, <a href="https://github.com/nix-community/nixos-anywhere">nixos-anywhere</a> and <a href="https://github.com/nix-community/disko">disko</a>, Clan makes it possible to have collaborative infrastructure</strong>.</p>
|
||||
<p>At the heart of Clan are <a href="./reference/clanServices/index.md">Clan Services</a> - the core concept that enables you to add functionality across multiple machines in your network. While Clan ships with essential core services, you can <a href="./guides/inventory/clanServices.md">create custom services</a> tailored to your specific needs.</p>
|
||||
|
||||
Reference in New Issue
Block a user