clan-core/docs/admins/backup.md

Backups

When self-hosting services, it's important to have a robust backup and restore strategy. Therefore clan comes with a backup integration based on borgbackup. More backup backends may come in future as clan provides an interchangeable interface on top of the backup implementation.

Getting started with borgbackup

Borgbackup consists of two components a backup repository that can be hosted on one machine and contains the backup and a backup client that will push it's data to the backup repository.

Borgbackup client

First you need to specify the remote server to backup to. Replace hostname with a reachable dns or ip address of your backup machine.

{
  clan.borgbackup.destinations = {
    myhostname = {
      repo = "borg@backuphost:/var/lib/borgbackup/myhostname";
    };
  };

  programs.ssh.knownHosts = {
    machine.hostNames = [ "backuphost" ];
    machine.publicKey = builtins.readFile ./machines/backuphost/facts/ssh.id_ed25519.pub;
  };
}

Services in clan can specify custom folders that need a backup by setting clanCore.state.<service>.folders option. As a user you can also append to the list by adding your own directories to be backed up i.e.:

{ clanCore.state.userdata.folders = [ "/home" "/root" ]; }

Than run clan secrets generate <yourmachine> replacing <yourmachine> with the actual machine name. This will generate the backup borg credentials and ssh keys for accessing the borgbackup repository. Your ssh public key will be stored in the root of the repository here at this location ./machines/<yourmachine>/facts/borgbackup.ssh.pub. We need this for the next step.

Borgbackup repository

In the next step we are going to set up the backup server. Choose here a machine with sufficient disk space. The machine needs to have the ssh daemon enabled as it is used in borgbackup for accessing the backup repository. Add the following configuration to your backup server:

{
  imports = [ inputs.clan-core.clanModules.sshd ];
  services.borgbackup.repos = {
    myhostname = {
      path = "/var/lib/borgbackup/myhostname";
      authorizedKeys = [
        (builtins.readFile ./machines/myhostname/facts/borgbackup.ssh.pub)
      ];
    };
  };
}

Replace myhostname with the name of the machine you want to backup. The path to the public key needs to be relative to the configuration file, so you may have to adapt it if the configuration is not in the root directory of your clan flake.

Afterwards run clan machines update to update both the borgbackup server and the borgbackup client.

By default the backup is scheduled every night at 01:00 midnight. If machines are not online around this time, they will attempt to run the backup once they come back.

When the next backup is scheduled, can be inspected like this on the machine running the backups

$ systemctl list-timers | grep -E 'NEXT|borg'
NEXT                                   LEFT LAST                              PASSED UNIT                               ACTIVATES
Thu 2024-03-14 01:00:00 CET             17h Wed 2024-03-13 01:00:00 CET       6h ago borgbackup-job-myhostname.timer borgbackup-job-myhostname.service

One can also list existing backups in the clan-cli

$ clan backups list mymachine
mymachine-mymachine-2024-03-09T01:00:00
mymachine-mymachine-2024-03-13T01:00:00

as well as triggering a manual backup:

$ clan backups create mymachine
[mymachine] $ bash -c systemctl start borgbackup-job-mymachine
successfully started backup