init

Use monospace font for code blocks.

Fixes: #1337

.env

@@ -1 +0,0 @@
-export OPENAI_API_KEY=$(rbw get openai-api-key)

.env.template

@@ -1 +0,0 @@
-export OPENAI_API_KEY=$(rbw get openai-api-key)

.envrc

@@ -2,11 +2,7 @@ if ! has nix_direnv_version || ! nix_direnv_version 3.0.4; then
   source_url "https://raw.githubusercontent.com/nix-community/nix-direnv/3.0.4/direnvrc" "sha256-DzlYZ33mWF/Gs8DDeyjr8mnVmQGx7ASYqA5WlxwvBG4="
 fi
 
-watch_file .direnv/selected-shell
-
-if [ -e .env ]; then
-  source .env
-fi
+watch_file .direnv/selected-shell 
 
 if [ -e .direnv/selected-shell ]; then
   use flake .#$(cat .direnv/selected-shell)

.gitea/workflows/checks.yaml

@@ -5,16 +5,6 @@ on:
     branches:
       - main
 jobs:
-  checks:
-    runs-on: nix
-    steps:
-      - uses: actions/checkout@v3
-      - run: nix run --refresh github:Mic92/nix-fast-build -- --no-nom --eval-workers 20
-  check-links:
-    runs-on: nix
-    steps:
-      - uses: actions/checkout@v3
-      - run: nix run --refresh --inputs-from .# nixpkgs#lychee .
   checks-impure:
     runs-on: nix
     steps:

.gitea/workflows/deploy.yaml

@@ -0,0 +1,13 @@
+name: deploy
+on:
+ push:
+    branches:
+      - main
+jobs:
+  deploy-docs:
+    runs-on: nix
+    steps:
+      - uses: actions/checkout@v3
+      - run: nix run .#deploy-docs
+    env:
+      SSH_HOMEPAGE_KEY: ${{ secrets.SSH_HOMEPAGE_KEY }}

.gitignore

@@ -9,9 +9,12 @@ example_clan
 result*
 /pkgs/clan-cli/clan_cli/nixpkgs
 /pkgs/clan-cli/clan_cli/webui/assets
-/machines
 nixos.qcow2
 **/*.glade~
+/docs/out
+
+# dream2nix
+.dream2nix
 
 # python
 __pycache__
@@ -21,3 +24,15 @@ __pycache__
 .reports
 .ruff_cache
 htmlcov
+
+# flatpak
+.flatpak-builder
+build
+build-dir
+repo
+.env
+
+# node
+node_modules
+dist
+.webui

CONTRIBUTING.md

@@ -0,0 +1,21 @@
+# Contributing to cLAN
+
+## Live-reloading documentation
+
+Enter the `docs` directory:
+
+```shell-session
+cd docs
+```
+
+Enter the development shell or enable `direnv`:
+
+```shell-session
+direnv allow
+```
+
+Run a local server:
+
+```shell-session
+mkdocs serve
+```

README.md

@@ -1,28 +1,45 @@
-# cLAN Core Repository
+# Clan Core Repository
 
-Welcome to the cLAN Core Repository, the heart of the [clan.lol](https://clan.lol/) project! This monorepo houses all the essential packages, NixOS modules, CLI tools, and tests you need to contribute and work with the cLAN project.
+Welcome to the Clan Core Repository, the heart of the [clan.lol](https://clan.lol/) project! This monorepo is the foundation of Clan, a revolutionary open-source project aimed at restoring fun, freedom, and functionality to computing. Here, you'll find all the essential packages, NixOS modules, CLI tools, and tests needed to contribute to and work with the cLAN project. Clan leverages the Nix system to ensure reliability, security, and seamless management of digital environments, putting the power back into the hands of users.
 
-## Getting Started
+## Why Clan?
 
-If you're new to cLAN and eager to dive in, start with our quickstart guide:
+Our mission is simple: to democratize computing by providing tools that empower users, foster innovation, and challenge outdated paradigms. Clan represents our contribution to a future where technology serves humanity, not the other way around. By participating in Clan, you're joining a movement dedicated to creating a secure, user-empowered digital future.
 
-- **Quickstart Guide**: Check out [quickstart.md](docs/admins/quickstart.md) to get up and running with cLAN in no time.
+## Features of Clan
 
-## Managing Secrets
+- **Full-Stack System Deployment:** Utilize Clan’s toolkit alongside Nix's reliability to build and manage systems effortlessly.
+- **Overlay Networks:** Secure, private communication channels between devices.
+- **Virtual Machine Integration:** Seamless operation of VM applications within the main operating system.
+- **Robust Backup Management:** Long-term, self-hosted data preservation.
+- **Intuitive Secret Management:** Simplified encryption and password management processes.
 
-Security is paramount, and cLAN provides guidelines for handling secrets effectively:
+## Getting Started with Clan
 
-- **Secrets Management**: Learn how to manage secrets securely by reading [secrets-management.md](docs/admins/secrets-management.md).
+If you're new to Clan and eager to dive in, start with our quickstart guide and explore the core functionalities that Clan offers:
 
-## Contributing to cLAN
+- **Quickstart Guide**: Check out [getting started](https://docs.clan.lol/#starting-with-a-new-clan-project)<!-- [docs/site/index.md](docs/site/index.md) --> to get up and running with Clan in no time.
 
-We welcome contributions from the community, and we've prepared a comprehensive guide to help you get started:
+### Managing Secrets
 
-- **Contribution Guidelines**: Find out how to contribute and make a meaningful impact on the cLAN project by reading [contributing.md](docs/contributing/contributing.md).
+In the Clan ecosystem, security is paramount. Learn how to handle secrets effectively:
 
-Whether you're a newcomer or a seasoned developer, we look forward to your contributions and collaboration on the cLAN project. Let's build amazing things together!
+- **Secrets Management**: Securely manage secrets by consulting [secrets](https://docs.clan.lol/getting-started/secrets/)<!-- [secrets.md](docs/site/getting-started/secrets.md) -->.
 
+### Contributing to Clan
+
+The Clan project thrives on community contributions. We welcome everyone to contribute and collaborate:
+
+- **Contribution Guidelines**: Make a meaningful impact by following the steps in [contributing](https://docs.clan.lol/contributing/contributing/)<!-- [contributing.md](docs/CONTRIBUTING.md) -->.
+
+## Join the Revolution
+
+Clan is more than a tool; it's a movement towards a better digital future. By contributing to the Clan project, you're part of changing technology for the better, together.
+
+### Community and Support
+
+Connect with us and the Clan community for support and discussion:
+
+- [Matrix channel](https://matrix.to/#/#clan:lassul.us) for live discussions.
+- IRC bridges (coming soon) for real-time chat support.
 
-### development environment
-Setup `direnv` and `nix-direnv` and execute `dienv allow`.
-To switch between different dev environments execute `select-shell`.

checks/backups/flake-module.nix

@@ -1,53 +1,58 @@
 { self, ... }:
-let
-  clan = self.lib.buildClan {
-    clanName = "testclan";
-    directory = ../..;
-    machines = {
-      test_backup_client = {
-        clan.networking.targetHost = "client";
-        imports = [ self.nixosModules.test_backup_client ];
-        fileSystems."/".device = "/dev/null";
-        boot.loader.grub.device = "/dev/null";
-      };
-    };
-  };
-in
 {
-  flake.nixosConfigurations = { inherit (clan.nixosConfigurations) test_backup_client; };
-  flake.clanInternals = clan.clanInternals;
+  clan.machines.test-backup = {
+    imports = [ self.nixosModules.test-backup ];
+    fileSystems."/".device = "/dev/null";
+    boot.loader.grub.device = "/dev/null";
+  };
   flake.nixosModules = {
-    test_backup_server = { ... }: {
-      imports = [
-        self.clanModules.borgbackup
-      ];
-      services.sshd.enable = true;
-      services.borgbackup.repos.testrepo = {
-        authorizedKeys = [
-          (builtins.readFile ../lib/ssh/pubkey)
-        ];
-      };
-    };
-    test_backup_client = { pkgs, lib, config, ... }:
+    test-backup =
+      {
+        pkgs,
+        lib,
+        config,
+        ...
+      }:
       let
         dependencies = [
           self
           pkgs.stdenv.drvPath
-          clan.clanInternals.machines.x86_64-linux.test_backup_client.config.system.clan.deployment.file
+          self.clanInternals.machines.${pkgs.hostPlatform.system}.test-backup.config.system.clan.deployment.file
         ] ++ builtins.map (i: i.outPath) (builtins.attrValues self.inputs);
         closureInfo = pkgs.closureInfo { rootPaths = dependencies; };
       in
       {
         imports = [
           self.clanModules.borgbackup
+          self.clanModules.localbackup
+          self.clanModules.sshd
         ];
-        networking.hostName = "client";
-        services.sshd.enable = true;
-        users.users.root.openssh.authorizedKeys.keyFiles = [
-          ../lib/ssh/pubkey
-        ];
+        clan.networking.targetHost = "machine";
+        networking.hostName = "machine";
+        services.openssh.settings.UseDns = false;
+
+        programs.ssh.knownHosts = {
+          machine.hostNames = [ "machine" ];
+          machine.publicKey = builtins.readFile ../lib/ssh/pubkey;
+        };
+
+        users.users.root.openssh.authorizedKeys.keyFiles = [ ../lib/ssh/pubkey ];
 
         systemd.tmpfiles.settings."vmsecrets" = {
+          "/root/.ssh/id_ed25519" = {
+            C.argument = "${../lib/ssh/privkey}";
+            z = {
+              mode = "0400";
+              user = "root";
+            };
+          };
+          "/etc/secrets/ssh.id_ed25519" = {
+            C.argument = "${../lib/ssh/privkey}";
+            z = {
+              mode = "0400";
+              user = "root";
+            };
+          };
           "/etc/secrets/borgbackup.ssh" = {
             C.argument = "${../lib/ssh/privkey}";
             z = {
@@ -63,10 +68,18 @@ in
             };
           };
         };
-        clanCore.secretStore = "vm";
+        clanCore.facts.secretStore = "vm";
 
-        environment.systemPackages = [ self.packages.${pkgs.system}.clan-cli ];
-        environment.etc."install-closure".source = "${closureInfo}/store-paths";
+        environment.systemPackages = [
+          self.packages.${pkgs.system}.clan-cli
+          (pkgs.writeShellScriptBin "pre-restore-command" ''
+            touch /var/test-service/pre-restore-command
+          '')
+          (pkgs.writeShellScriptBin "post-restore-command" ''
+            touch /var/test-service/post-restore-command
+          '')
+        ];
+        environment.etc.install-closure.source = "${closureInfo}/store-paths";
         nix.settings = {
           substituters = lib.mkForce [ ];
           hashed-mirrors = null;
@@ -75,71 +88,91 @@ in
         };
         system.extraDependencies = dependencies;
         clanCore.state.test-backups.folders = [ "/var/test-backups" ];
-        clan.borgbackup = {
-          enable = true;
-          destinations.test_backup_server.repo = "borg@server:.";
+
+        clanCore.state.test-service = {
+          preRestoreCommand = "pre-restore-command";
+          postRestoreCommand = "post-restore-command";
+          folders = [ "/var/test-service" ];
+        };
+        clan.borgbackup.destinations.test-backup.repo = "borg@machine:.";
+
+        fileSystems."/mnt/external-disk" = {
+          device = "/dev/vdb"; # created in tests with virtualisation.emptyDisks
+          autoFormat = true;
+          fsType = "ext4";
+          options = [
+            "defaults"
+            "noauto"
+          ];
+        };
+
+        clan.localbackup.targets.hdd = {
+          directory = "/mnt/external-disk";
+          preMountHook = ''
+            touch /run/mount-external-disk
+          '';
+          postUnmountHook = ''
+            touch /run/unmount-external-disk
+          '';
+        };
+
+        services.borgbackup.repos.test-backups = {
+          path = "/var/lib/borgbackup/test-backups";
+          authorizedKeys = [ (builtins.readFile ../lib/ssh/pubkey) ];
         };
       };
   };
-  perSystem = { nodes, pkgs, ... }: {
-    checks = pkgs.lib.mkIf (pkgs.stdenv.isLinux) {
-      test-backups =
-        (import ../lib/test-base.nix)
-          {
-            name = "test-backups";
-            nodes.server = {
-              imports = [
-                self.nixosModules.test_backup_server
-                self.nixosModules.clanCore
-                {
-                  clanCore.machineName = "server";
-                  clanCore.clanDir = ../..;
-                }
-              ];
-            };
-            nodes.client = {
-              imports = [
-                self.nixosModules.test_backup_client
-                self.nixosModules.clanCore
-                {
-                  clanCore.machineName = "client";
-                  clanCore.clanDir = ../..;
-                }
-              ];
-            };
+  perSystem =
+    { nodes, pkgs, ... }:
+    {
+      checks = pkgs.lib.mkIf (pkgs.stdenv.isLinux) {
+        test-backups = (import ../lib/test-base.nix) {
+          name = "test-backups";
+          nodes.machine = {
+            imports = [
+              self.nixosModules.clanCore
+              self.nixosModules.test-backup
+            ];
+            virtualisation.emptyDiskImages = [ 256 ];
+          };
 
-            testScript = ''
-              import json
-              start_all()
+          testScript = ''
+            import json
+            start_all()
 
-              # setup
-              client.succeed("mkdir -m 700 /root/.ssh")
-              client.succeed(
-                  "cat ${../lib/ssh/privkey} > /root/.ssh/id_ed25519"
-              )
-              client.succeed("chmod 600 /root/.ssh/id_ed25519")
-              client.wait_for_unit("sshd", timeout=30)
-              client.succeed("ssh -o StrictHostKeyChecking=accept-new root@client hostname")
+            # dummy data
+            machine.succeed("mkdir -p /var/test-backups /var/test-service")
+            machine.succeed("echo testing > /var/test-backups/somefile")
 
-              # dummy data
-              client.succeed("mkdir /var/test-backups")
-              client.succeed("echo testing > /var/test-backups/somefile")
+            # create
+            machine.succeed("clan --debug --flake ${self} backups create test-backup")
+            machine.wait_until_succeeds("! systemctl is-active borgbackup-job-test-backup >&2")
+            machine.succeed("test -f /run/mount-external-disk")
+            machine.succeed("test -f /run/unmount-external-disk")
 
-              # create
-              client.succeed("clan --debug --flake ${../..} backups create test_backup_client")
-              client.wait_until_succeeds("! systemctl is-active borgbackup-job-test_backup_server")
+            # list
+            backup_id = json.loads(machine.succeed("borg-job-test-backup list --json"))["archives"][0]["archive"]
+            out = machine.succeed("clan --debug --flake ${self} backups list test-backup").strip()
+            print(out)
+            assert backup_id in out, f"backup {backup_id} not found in {out}"
+            localbackup_id = "hdd::/mnt/external-disk/snapshot.0"
+            assert localbackup_id in out, "localbackup not found in {out}"
 
-              # list
-              backup_id = json.loads(client.succeed("borg-job-test_backup_server list --json"))["archives"][0]["archive"]
-              assert(backup_id in client.succeed("clan --debug --flake ${../..} backups list test_backup_client"))
+            ## borgbackup restore
+            machine.succeed("rm -f /var/test-backups/somefile")
+            machine.succeed(f"clan --debug --flake ${self} backups restore test-backup borgbackup 'test-backup::borg@machine:.::{backup_id}' >&2")
+            assert machine.succeed("cat /var/test-backups/somefile").strip() == "testing", "restore failed"
+            machine.succeed("test -f /var/test-service/pre-restore-command")
+            machine.succeed("test -f /var/test-service/post-restore-command")
 
-              # restore
-              client.succeed("rm -f /var/test-backups/somefile")
-              client.succeed(f"clan --debug --flake ${../..} backups restore test_backup_client borgbackup {backup_id}")
-              assert(client.succeed("cat /var/test-backups/somefile").strip() == "testing")
-            '';
-          }
-          { inherit pkgs self; };
+            ## localbackup restore
+            machine.succeed("rm -f /var/test-backups/somefile /var/test-service/{pre,post}-restore-command")
+            machine.succeed(f"clan --debug --flake ${self} backups restore test-backup localbackup '{localbackup_id}' >&2")
+            assert machine.succeed("cat /var/test-backups/somefile").strip() == "testing", "restore failed"
+            machine.succeed("test -f /var/test-service/pre-restore-command")
+            machine.succeed("test -f /var/test-service/post-restore-command")
+          '';
+        } { inherit pkgs self; };
+      };
     };
-  };
 }

checks/borgbackup/default.nix

@@ -1,51 +1,51 @@
-(import ../lib/test-base.nix) ({ ... }: {
-  name = "borgbackup";
+(import ../lib/test-base.nix) (
+  { ... }:
+  {
+    name = "borgbackup";
 
-  nodes.machine = { self, pkgs, ... }: {
-    imports = [
-      self.clanModules.borgbackup
-      self.nixosModules.clanCore
+    nodes.machine =
+      { self, pkgs, ... }:
       {
-        services.openssh.enable = true;
-        services.borgbackup.repos.testrepo = {
-          authorizedKeys = [
-            (builtins.readFile ../lib/ssh/pubkey)
-          ];
-        };
-      }
-      {
-        clanCore.machineName = "machine";
-        clanCore.clanDir = ./.;
-        clanCore.state.testState.folders = [ "/etc/state" ];
-        environment.etc.state.text = "hello world";
-        systemd.tmpfiles.settings."vmsecrets" = {
-          "/etc/secrets/borgbackup.ssh" = {
-            C.argument = "${../lib/ssh/privkey}";
-            z = {
-              mode = "0400";
-              user = "root";
+        imports = [
+          self.clanModules.borgbackup
+          self.nixosModules.clanCore
+          {
+            services.openssh.enable = true;
+            services.borgbackup.repos.testrepo = {
+              authorizedKeys = [ (builtins.readFile ../lib/ssh/pubkey) ];
             };
-          };
-          "/etc/secrets/borgbackup.repokey" = {
-            C.argument = builtins.toString (pkgs.writeText "repokey" "repokey12345");
-            z = {
-              mode = "0400";
-              user = "root";
+          }
+          {
+            clanCore.machineName = "machine";
+            clanCore.clanDir = ./.;
+            clanCore.state.testState.folders = [ "/etc/state" ];
+            environment.etc.state.text = "hello world";
+            systemd.tmpfiles.settings."vmsecrets" = {
+              "/etc/secrets/borgbackup.ssh" = {
+                C.argument = "${../lib/ssh/privkey}";
+                z = {
+                  mode = "0400";
+                  user = "root";
+                };
+              };
+              "/etc/secrets/borgbackup.repokey" = {
+                C.argument = builtins.toString (pkgs.writeText "repokey" "repokey12345");
+                z = {
+                  mode = "0400";
+                  user = "root";
+                };
+              };
             };
-          };
-        };
-        clanCore.secretStore = "vm";
+            clanCore.facts.secretStore = "vm";
 
-        clan.borgbackup = {
-          enable = true;
-          destinations.test.repo = "borg@localhost:.";
-        };
-      }
-    ];
-  };
-  testScript = ''
-    start_all()
-    machine.systemctl("start --wait borgbackup-job-test.service")
-    assert "machine-test" in machine.succeed("BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes /run/current-system/sw/bin/borg-job-test list")
-  '';
-})
+            clan.borgbackup.destinations.test.repo = "borg@localhost:.";
+          }
+        ];
+      };
+    testScript = ''
+      start_all()
+      machine.systemctl("start --wait borgbackup-job-test.service")
+      assert "machine-test" in machine.succeed("BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes /run/current-system/sw/bin/borg-job-test list")
+    '';
+  }
+)

checks/container/default.nix

@@ -1,14 +1,19 @@
-(import ../lib/container-test.nix) ({ ... }: {
-  name = "secrets";
+(import ../lib/container-test.nix) (
+  { ... }:
+  {
+    name = "secrets";
 
-  nodes.machine = { ... }: {
-    networking.hostName = "machine";
-    services.openssh.enable = true;
-    services.openssh.startWhenNeeded = false;
-  };
-  testScript = ''
-    start_all()
-    machine.succeed("systemctl status sshd")
-    machine.wait_for_unit("sshd")
-  '';
-})
+    nodes.machine =
+      { ... }:
+      {
+        networking.hostName = "machine";
+        services.openssh.enable = true;
+        services.openssh.startWhenNeeded = false;
+      };
+    testScript = ''
+      start_all()
+      machine.succeed("systemctl status sshd")
+      machine.wait_for_unit("sshd")
+    '';
+  }
+)

checks/deltachat/default.nix

@@ -1,24 +1,29 @@
-(import ../lib/container-test.nix) ({ pkgs, ... }: {
-  name = "secrets";
+(import ../lib/container-test.nix) (
+  { pkgs, ... }:
+  {
+    name = "secrets";
 
-  nodes.machine = { self, ... }: {
-    imports = [
-      self.clanModules.deltachat
-      self.nixosModules.clanCore
+    nodes.machine =
+      { self, ... }:
       {
-        clanCore.machineName = "machine";
-        clanCore.clanDir = ./.;
-      }
-    ];
-  };
-  testScript = ''
-    start_all()
-    machine.wait_for_unit("maddy")
-    # imap
-    machine.succeed("${pkgs.netcat}/bin/nc -z -v ::1 143")
-    # smtp submission
-    machine.succeed("${pkgs.netcat}/bin/nc -z -v ::1 587")
-    # smtp
-    machine.succeed("${pkgs.netcat}/bin/nc -z -v ::1 25")
-  '';
-})
+        imports = [
+          self.clanModules.deltachat
+          self.nixosModules.clanCore
+          {
+            clanCore.machineName = "machine";
+            clanCore.clanDir = ./.;
+          }
+        ];
+      };
+    testScript = ''
+      start_all()
+      machine.wait_for_unit("maddy")
+      # imap
+      machine.succeed("${pkgs.netcat}/bin/nc -z -v ::1 143")
+      # smtp submission
+      machine.succeed("${pkgs.netcat}/bin/nc -z -v ::1 587")
+      # smtp
+      machine.succeed("${pkgs.netcat}/bin/nc -z -v ::1 25")
+    '';
+  }
+)

checks/flake-module.nix

@@ -1,54 +1,78 @@
-{ self, ... }: {
+{ self, ... }:
+{
   imports = [
     ./impure/flake-module.nix
     ./backups/flake-module.nix
     ./installation/flake-module.nix
+    ./flash/flake-module.nix
   ];
-  perSystem = { pkgs, lib, self', ... }: {
-    checks =
-      let
-        nixosTestArgs = {
-          # reference to nixpkgs for the current system
-          inherit pkgs;
-          # this gives us a reference to our flake but also all flake inputs
-          inherit self;
-        };
-        nixosTests = lib.optionalAttrs (pkgs.stdenv.isLinux) {
-          # import our test
-          secrets = import ./secrets nixosTestArgs;
-          container = import ./container nixosTestArgs;
-          deltachat = import ./deltachat nixosTestArgs;
-          meshnamed = import ./meshnamed nixosTestArgs;
-          zt-tcp-relay = import ./zt-tcp-relay nixosTestArgs;
-          borgbackup = import ./borgbackup nixosTestArgs;
-          syncthing = import ./syncthing nixosTestArgs;
-          wayland-proxy-virtwl = import ./wayland-proxy-virtwl nixosTestArgs;
-        };
-        schemaTests = pkgs.callPackages ./schemas.nix {
-          inherit self;
-        };
-
-        flakeOutputs = lib.mapAttrs' (name: config: lib.nameValuePair "nixos-${name}" config.config.system.build.toplevel) self.nixosConfigurations
-          // lib.mapAttrs' (n: lib.nameValuePair "package-${n}") self'.packages
-          // lib.mapAttrs' (n: lib.nameValuePair "devShell-${n}") self'.devShells
-          // lib.mapAttrs' (name: config: lib.nameValuePair "home-manager-${name}" config.activation-script) (self'.legacyPackages.homeConfigurations or { });
-      in
-      nixosTests // schemaTests // flakeOutputs;
-    legacyPackages = {
-      nixosTests =
+  perSystem =
+    {
+      pkgs,
+      lib,
+      self',
+      ...
+    }:
+    {
+      checks =
         let
+          # ensure all options can be rendered after importing clan into nixos
+          renderClanOptions =
+            let
+              docs = pkgs.nixosOptionsDoc {
+                options =
+                  (pkgs.nixos {
+                    imports = [ self.nixosModules.clanCore ];
+                    clanCore.clanDir = ./.;
+                  }).options;
+                warningsAreErrors = false;
+              };
+            in
+            docs.optionsJSON;
           nixosTestArgs = {
             # reference to nixpkgs for the current system
             inherit pkgs;
             # this gives us a reference to our flake but also all flake inputs
             inherit self;
           };
+          nixosTests = lib.optionalAttrs (pkgs.stdenv.isLinux) {
+            # import our test
+            secrets = import ./secrets nixosTestArgs;
+            container = import ./container nixosTestArgs;
+            deltachat = import ./deltachat nixosTestArgs;
+            matrix-synapse = import ./matrix-synapse nixosTestArgs;
+            zt-tcp-relay = import ./zt-tcp-relay nixosTestArgs;
+            borgbackup = import ./borgbackup nixosTestArgs;
+            syncthing = import ./syncthing nixosTestArgs;
+            wayland-proxy-virtwl = import ./wayland-proxy-virtwl nixosTestArgs;
+          };
+
+          flakeOutputs =
+            lib.mapAttrs' (
+              name: config: lib.nameValuePair "nixos-${name}" config.config.system.build.toplevel
+            ) self.nixosConfigurations
+            // lib.mapAttrs' (n: lib.nameValuePair "package-${n}") self'.packages
+            // lib.mapAttrs' (n: lib.nameValuePair "devShell-${n}") self'.devShells
+            // lib.mapAttrs' (name: config: lib.nameValuePair "home-manager-${name}" config.activation-script) (
+              self'.legacyPackages.homeConfigurations or { }
+            );
         in
-        lib.optionalAttrs (pkgs.stdenv.isLinux) {
-          # import our test
-          secrets = import ./secrets nixosTestArgs;
-          container = import ./container nixosTestArgs;
-        };
+        { inherit renderClanOptions; } // nixosTests // flakeOutputs;
+      legacyPackages = {
+        nixosTests =
+          let
+            nixosTestArgs = {
+              # reference to nixpkgs for the current system
+              inherit pkgs;
+              # this gives us a reference to our flake but also all flake inputs
+              inherit self;
+            };
+          in
+          lib.optionalAttrs (pkgs.stdenv.isLinux) {
+            # import our test
+            secrets = import ./secrets nixosTestArgs;
+            container = import ./container nixosTestArgs;
+          };
+      };
     };
-  };
 }

checks/flash/flake-module.nix

@@ -0,0 +1,33 @@
+{ ... }:
+{
+  perSystem =
+    { ... }:
+    {
+      # checks = pkgs.lib.mkIf (pkgs.stdenv.isLinux) {
+      #   flash = (import ../lib/test-base.nix) {
+      #     name = "flash";
+      #     nodes.target = {
+      #       virtualisation.emptyDiskImages = [ 4096 ];
+      #       virtualisation.memorySize = 3000;
+      #       environment.systemPackages = [ self.packages.${pkgs.system}.clan-cli ];
+      #       environment.etc."install-closure".source = "${closureInfo}/store-paths"; 
+
+      #       nix.settings = {
+      #         substituters = lib.mkForce [ ];
+      #         hashed-mirrors = null;
+      #         connect-timeout = lib.mkForce 3;
+      #         flake-registry = pkgs.writeText "flake-registry" ''{"flakes":[],"version":2}'';
+      #         experimental-features = [
+      #           "nix-command"
+      #           "flakes"
+      #         ];
+      #       };
+      #     };
+      #     testScript = ''
+      #       start_all()
+      #       machine.succeed("clan --debug --flake ${../..} flash --yes --disk main /dev/vdb test_install_machine")
+      #     '';
+      #   } { inherit pkgs self; };
+      # };
+    };
+}

checks/impure/flake-module.nix

@@ -1,18 +1,24 @@
 {
-  perSystem = { pkgs, lib, ... }: {
-    # a script that executes all other checks
-    packages.impure-checks = pkgs.writeShellScriptBin "impure-checks" ''
-      #!${pkgs.bash}/bin/bash
-      set -euo pipefail
+  perSystem =
+    { pkgs, lib, ... }:
+    {
+      # a script that executes all other checks
+      packages.impure-checks = pkgs.writeShellScriptBin "impure-checks" ''
+        #!${pkgs.bash}/bin/bash
+        set -euo pipefail
 
-      export PATH="${lib.makeBinPath [
-        pkgs.gitMinimal
-        pkgs.nix
-        pkgs.rsync # needed to have rsync installed on the dummy ssh server
-      ]}"
-      ROOT=$(git rev-parse --show-toplevel)
-      cd "$ROOT/pkgs/clan-cli"
-      nix develop "$ROOT#clan-cli" -c bash -c "TMPDIR=/tmp python -m pytest -s -m impure ./tests $@"
-    '';
-  };
+        unset CLAN_DIR
+
+        export PATH="${
+          lib.makeBinPath [
+            pkgs.gitMinimal
+            pkgs.nix
+            pkgs.rsync # needed to have rsync installed on the dummy ssh server
+          ]
+        }"
+        ROOT=$(git rev-parse --show-toplevel)
+        cd "$ROOT/pkgs/clan-cli"
+        nix develop "$ROOT#clan-cli" -c bash -c "TMPDIR=/tmp python -m pytest -s -m impure ./tests $@"
+      '';
+    };
 }

checks/installation/flake-module.nix

@@ -1,124 +1,114 @@
-{ self, ... }:
-let
-  clan = self.lib.buildClan {
-    clanName = "testclan";
-    directory = ../..;
-    machines = {
-      test_install_machine = {
-        clan.networking.targetHost = "test_install_machine";
-        imports = [ self.nixosModules.test_install_machine ];
-      };
-    };
-  };
-in
+{ self, lib, ... }:
 {
-  flake.nixosConfigurations = { inherit (clan.nixosConfigurations) test_install_machine; };
-  flake.clanInternals = clan.clanInternals;
-  flake.nixosModules = {
-    test_install_machine = { lib, modulesPath, ... }: {
-      imports = [
-        self.clanModules.diskLayouts
-        (modulesPath + "/testing/test-instrumentation.nix") # we need these 2 modules always to be able to run the tests
-        (modulesPath + "/profiles/qemu-guest.nix")
-      ];
-      fileSystems."/nix/store" = lib.mkForce {
-        device = "nix-store";
-        fsType = "9p";
-        neededForBoot = true;
-        options = [ "trans=virtio" "version=9p2000.L" "cache=loose" ];
-      };
-      clan.diskLayouts.singleDiskExt4.device = "/dev/vdb";
+  clan.machines.test_install_machine = {
+    clan.networking.targetHost = "test_install_machine";
+    fileSystems."/".device = lib.mkDefault "/dev/null";
+    boot.loader.grub.device = lib.mkDefault "/dev/null";
 
-      environment.etc."install-successful".text = "ok";
-
-      boot.consoleLogLevel = lib.mkForce 100;
-      boot.kernelParams = [
-        "boot.shell_on_fail"
-      ];
-    };
+    imports = [ self.nixosModules.test_install_machine ];
   };
-  perSystem = { nodes, pkgs, lib, ... }:
+  flake.nixosModules = {
+    test_install_machine =
+      { lib, modulesPath, ... }:
+      {
+        imports = [
+          self.clanModules.disk-layouts
+          (modulesPath + "/testing/test-instrumentation.nix") # we need these 2 modules always to be able to run the tests
+          (modulesPath + "/profiles/qemu-guest.nix")
+        ];
+        clan.disk-layouts.singleDiskExt4.device = "/dev/vdb";
+
+        environment.etc."install-successful".text = "ok";
+
+        boot.consoleLogLevel = lib.mkForce 100;
+        boot.kernelParams = [ "boot.shell_on_fail" ];
+      };
+  };
+  perSystem =
+    {
+      nodes,
+      pkgs,
+      lib,
+      ...
+    }:
     let
       dependencies = [
         self
+        self.nixosConfigurations.test_install_machine.config.system.build.toplevel
+        self.nixosConfigurations.test_install_machine.config.system.build.diskoScript
+        self.nixosConfigurations.test_install_machine.config.system.clan.deployment.file
         pkgs.stdenv.drvPath
-        clan.clanInternals.machines.x86_64-linux.test_install_machine.config.system.build.toplevel
-        clan.clanInternals.machines.x86_64-linux.test_install_machine.config.system.build.diskoScript
-        clan.clanInternals.machines.x86_64-linux.test_install_machine.config.system.clan.deployment.file
         pkgs.nixos-anywhere
       ] ++ builtins.map (i: i.outPath) (builtins.attrValues self.inputs);
       closureInfo = pkgs.closureInfo { rootPaths = dependencies; };
     in
     {
       checks = pkgs.lib.mkIf (pkgs.stdenv.isLinux) {
-        test-installation =
-          (import ../lib/test-base.nix)
-            {
-              name = "test-installation";
-              nodes.target = {
-                services.openssh.enable = true;
-                users.users.root.openssh.authorizedKeys.keyFiles = [
-                  ../lib/ssh/pubkey
-                ];
-                system.nixos.variant_id = "installer";
-                virtualisation.emptyDiskImages = [ 4096 ];
-                nix.settings = {
-                  substituters = lib.mkForce [ ];
-                  hashed-mirrors = null;
-                  connect-timeout = lib.mkForce 3;
-                  flake-registry = pkgs.writeText "flake-registry" ''{"flakes":[],"version":2}'';
-                  experimental-features = [
-                    "nix-command"
-                    "flakes"
-                  ];
-                };
-              };
-              nodes.client = {
-                environment.systemPackages = [ self.packages.${pkgs.system}.clan-cli ];
-                environment.etc."install-closure".source = "${closureInfo}/store-paths";
-                virtualisation.memorySize = 2048;
-                nix.settings = {
-                  substituters = lib.mkForce [ ];
-                  hashed-mirrors = null;
-                  connect-timeout = lib.mkForce 3;
-                  flake-registry = pkgs.writeText "flake-registry" ''{"flakes":[],"version":2}'';
-                  experimental-features = [
-                    "nix-command"
-                    "flakes"
-                  ];
-                };
-                system.extraDependencies = dependencies;
-              };
+        test-installation = (import ../lib/test-base.nix) {
+          name = "test-installation";
+          nodes.target = {
+            services.openssh.enable = true;
+            users.users.root.openssh.authorizedKeys.keyFiles = [ ../lib/ssh/pubkey ];
+            system.nixos.variant_id = "installer";
+            virtualisation.emptyDiskImages = [ 4096 ];
+            nix.settings = {
+              substituters = lib.mkForce [ ];
+              hashed-mirrors = null;
+              connect-timeout = lib.mkForce 3;
+              flake-registry = pkgs.writeText "flake-registry" ''{"flakes":[],"version":2}'';
+              experimental-features = [
+                "nix-command"
+                "flakes"
+              ];
+            };
+          };
+          nodes.client = {
+            environment.systemPackages = [
+              self.packages.${pkgs.system}.clan-cli
+            ] ++ self.packages.${pkgs.system}.clan-cli.runtimeDependencies;
+            environment.etc."install-closure".source = "${closureInfo}/store-paths";
+            virtualisation.memorySize = 2048;
+            nix.settings = {
+              substituters = lib.mkForce [ ];
+              hashed-mirrors = null;
+              connect-timeout = lib.mkForce 3;
+              flake-registry = pkgs.writeText "flake-registry" ''{"flakes":[],"version":2}'';
+              experimental-features = [
+                "nix-command"
+                "flakes"
+              ];
+            };
+            system.extraDependencies = dependencies;
+          };
 
-              testScript = ''
-                def create_test_machine(oldmachine=None, args={}): # taken from <nixpkgs/nixos/tests/installer.nix>
-                    machine = create_machine({
-                      "qemuFlags":
-                        '-cpu max -m 1024 -virtfs local,path=/nix/store,security_model=none,mount_tag=nix-store,'
-                        f' -drive file={oldmachine.state_dir}/empty0.qcow2,id=drive1,if=none,index=1,werror=report'
-                        f' -device virtio-blk-pci,drive=drive1',
-                    } | args)
-                    driver.machines.append(machine)
-                    return machine
+          testScript = ''
+            def create_test_machine(oldmachine=None, args={}): # taken from <nixpkgs/nixos/tests/installer.nix>
+                startCommand = "${pkgs.qemu_test}/bin/qemu-kvm"
+                startCommand += " -cpu max -m 1024 -virtfs local,path=/nix/store,security_model=none,mount_tag=nix-store"
+                startCommand += f' -drive file={oldmachine.state_dir}/empty0.qcow2,id=drive1,if=none,index=1,werror=report'
+                startCommand += ' -device virtio-blk-pci,drive=drive1'
+                machine = create_machine({
+                  "startCommand": startCommand,
+                } | args)
+                driver.machines.append(machine)
+                return machine
 
+            start_all()
 
-                start_all()
+            client.succeed("${pkgs.coreutils}/bin/install -Dm 600 ${../lib/ssh/privkey} /root/.ssh/id_ed25519")
+            client.wait_until_succeeds("ssh -o StrictHostKeyChecking=accept-new -v root@target hostname")
 
-                client.succeed("${pkgs.coreutils}/bin/install -Dm 600 ${../lib/ssh/privkey} /root/.ssh/id_ed25519")
-                client.wait_until_succeeds("ssh -o StrictHostKeyChecking=accept-new -v root@target hostname")
+            client.succeed("clan --debug --flake ${../..} machines install --yes test_install_machine root@target >&2")
+            try:
+              target.shutdown()
+            except BrokenPipeError:
+              # qemu has already exited
+              pass
 
-                client.succeed("clan --debug --flake ${../..} machines install test_install_machine root@target >&2")
-                try:
-                  target.shutdown()
-                except BrokenPipeError:
-                  # qemu has already exited
-                  pass
-
-                new_machine = create_test_machine(oldmachine=target, args={ "name": "new_machine" })
-                assert(new_machine.succeed("cat /etc/install-successful").strip() == "ok")
-              '';
-            }
-            { inherit pkgs self; };
+            new_machine = create_test_machine(oldmachine=target, args={ "name": "new_machine" })
+            assert(new_machine.succeed("cat /etc/install-successful").strip() == "ok")
+          '';
+        } { inherit pkgs self; };
       };
     };
 }

checks/lib/container-driver/module.nix

@@ -1,17 +1,23 @@
-{ hostPkgs, lib, config, ... }:
+{
+  hostPkgs,
+  lib,
+  config,
+  ...
+}:
 let
   testDriver = hostPkgs.python3.pkgs.callPackage ./package.nix {
     inherit (config) extraPythonPackages;
     inherit (hostPkgs.pkgs) util-linux systemd;
   };
   containers = map (m: m.system.build.toplevel) (lib.attrValues config.nodes);
-  pythonizeName = name:
+  pythonizeName =
+    name:
     let
       head = lib.substring 0 1 name;
       tail = lib.substring 1 (-1) name;
     in
-    (if builtins.match "[A-z_]" head == null then "_" else head) +
-    lib.stringAsChars (c: if builtins.match "[A-z0-9_]" c == null then "_" else c) tail;
+    (if builtins.match "[A-z_]" head == null then "_" else head)
+    + lib.stringAsChars (c: if builtins.match "[A-z0-9_]" c == null then "_" else c) tail;
   nodeHostNames =
     let
       nodesList = map (c: c.system.name) (lib.attrValues config.nodes);
@@ -21,68 +27,72 @@ let
   pythonizedNames = map pythonizeName nodeHostNames;
 in
 {
-  driver = lib.mkForce (hostPkgs.runCommand "nixos-test-driver-${config.name}"
-    {
-      nativeBuildInputs = [
-        hostPkgs.makeWrapper
-      ] ++ lib.optionals (!config.skipTypeCheck) [ hostPkgs.mypy ];
-      buildInputs = [ testDriver ];
-      testScript = config.testScriptString;
-      preferLocalBuild = true;
-      passthru = config.passthru;
-      meta = config.meta // {
-        mainProgram = "nixos-test-driver";
+  driver = lib.mkForce (
+    hostPkgs.runCommand "nixos-test-driver-${config.name}"
+      {
+        nativeBuildInputs = [
+          hostPkgs.makeWrapper
+        ] ++ lib.optionals (!config.skipTypeCheck) [ hostPkgs.mypy ];
+        buildInputs = [ testDriver ];
+        testScript = config.testScriptString;
+        preferLocalBuild = true;
+        passthru = config.passthru;
+        meta = config.meta // {
+          mainProgram = "nixos-test-driver";
+        };
+      }
+      ''
+        mkdir -p $out/bin
+
+        containers=(${toString containers})
+
+        ${lib.optionalString (!config.skipTypeCheck) ''
+          # prepend type hints so the test script can be type checked with mypy
+          cat "${./test-script-prepend.py}" >> testScriptWithTypes
+          echo "${builtins.toString machineNames}" >> testScriptWithTypes
+          echo -n "$testScript" >> testScriptWithTypes
+
+          echo "Running type check (enable/disable: config.skipTypeCheck)"
+          echo "See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck"
+
+          mypy  --no-implicit-optional \
+                --pretty \
+                --no-color-output \
+                testScriptWithTypes
+        ''}
+
+        echo -n "$testScript" >> $out/test-script
+
+        ln -s ${testDriver}/bin/nixos-test-driver $out/bin/nixos-test-driver
+
+        wrapProgram $out/bin/nixos-test-driver \
+          ${lib.concatStringsSep " " (map (name: "--add-flags '--container ${name}'") containers)} \
+          --add-flags "--test-script '$out/test-script'"
+      ''
+  );
+
+  test = lib.mkForce (
+    lib.lazyDerivation {
+      # lazyDerivation improves performance when only passthru items and/or meta are used.
+      derivation = hostPkgs.stdenv.mkDerivation {
+        name = "vm-test-run-${config.name}";
+
+        requiredSystemFeatures = [ "uid-range" ];
+
+        buildCommand = ''
+          mkdir -p $out
+
+          # effectively mute the XMLLogger
+          export LOGFILE=/dev/null
+
+          ${config.driver}/bin/nixos-test-driver -o $out
+        '';
+
+        passthru = config.passthru;
+
+        meta = config.meta;
       };
+      inherit (config) passthru meta;
     }
-    ''
-      mkdir -p $out/bin
-
-      containers=(${toString containers})
-
-      ${lib.optionalString (!config.skipTypeCheck) ''
-        # prepend type hints so the test script can be type checked with mypy
-        cat "${./test-script-prepend.py}" >> testScriptWithTypes
-        echo "${builtins.toString machineNames}" >> testScriptWithTypes
-        echo -n "$testScript" >> testScriptWithTypes
-
-        echo "Running type check (enable/disable: config.skipTypeCheck)"
-        echo "See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck"
-
-        mypy  --no-implicit-optional \
-              --pretty \
-              --no-color-output \
-              testScriptWithTypes
-      ''}
-
-      echo -n "$testScript" >> $out/test-script
-
-      ln -s ${testDriver}/bin/nixos-test-driver $out/bin/nixos-test-driver
-
-      wrapProgram $out/bin/nixos-test-driver \
-        ${lib.concatStringsSep " " (map (name: "--add-flags '--container ${name}'") containers)} \
-        --add-flags "--test-script '$out/test-script'"
-    '');
-
-  test = lib.mkForce (lib.lazyDerivation {
-    # lazyDerivation improves performance when only passthru items and/or meta are used.
-    derivation = hostPkgs.stdenv.mkDerivation {
-      name = "vm-test-run-${config.name}";
-
-      requiredSystemFeatures = [ "uid-range" ];
-
-      buildCommand = ''
-        mkdir -p $out
-
-        # effectively mute the XMLLogger
-        export LOGFILE=/dev/null
-
-        ${config.driver}/bin/nixos-test-driver -o $out
-      '';
-
-      passthru = config.passthru;
-
-      meta = config.meta;
-    };
-    inherit (config) passthru meta;
-  });
+  );
 }

checks/lib/container-driver/package.nix

@@ -1,8 +1,18 @@
-{ extraPythonPackages, python3Packages, buildPythonApplication, setuptools, util-linux, systemd }:
+{
+  extraPythonPackages,
+  python3Packages,
+  buildPythonApplication,
+  setuptools,
+  util-linux,
+  systemd,
+}:
 buildPythonApplication {
   pname = "test-driver";
   version = "0.0.1";
-  propagatedBuildInputs = [ util-linux systemd ] ++ extraPythonPackages python3Packages;
+  propagatedBuildInputs = [
+    util-linux
+    systemd
+  ] ++ extraPythonPackages python3Packages;
   nativeBuildInputs = [ setuptools ];
   format = "pyproject";
   src = ./.;

checks/lib/container-driver/test_driver/__init__.py

@@ -258,7 +258,7 @@ class Driver:
 
         self.machines = []
         for container in containers:
-            name_match = re.match(r".*-nixos-system-(.+)-\d.+", container.name)
+            name_match = re.match(r".*-nixos-system-(.+)-(.+)", container.name)
             if not name_match:
                 raise ValueError(f"Unable to extract hostname from {container.name}")
             name = name_match.group(1)

checks/lib/container-test.nix

@@ -1,33 +1,33 @@
 test:
-{ pkgs
-, self
-, ...
-}:
+{ pkgs, self, ... }:
 let
   inherit (pkgs) lib;
   nixos-lib = import (pkgs.path + "/nixos/lib") { };
 in
-(nixos-lib.runTest ({ hostPkgs, ... }: {
-  hostPkgs = pkgs;
-  # speed-up evaluation
-  defaults = {
-    documentation.enable = lib.mkDefault false;
-    boot.isContainer = true;
+(nixos-lib.runTest (
+  { hostPkgs, ... }:
+  {
+    hostPkgs = pkgs;
+    # speed-up evaluation
+    defaults = {
+      documentation.enable = lib.mkDefault false;
+      boot.isContainer = true;
 
-    # undo qemu stuff
-    system.build.initialRamdisk = "";
-    virtualisation.sharedDirectories = lib.mkForce { };
-    networking.useDHCP = false;
+      # undo qemu stuff
+      system.build.initialRamdisk = "";
+      virtualisation.sharedDirectories = lib.mkForce { };
+      networking.useDHCP = false;
 
-    # we have not private networking so far
-    networking.interfaces = lib.mkForce { };
-    #networking.primaryIPAddress = lib.mkForce null;
-    systemd.services.backdoor.enable = false;
-  };
-  # to accept external dependencies such as disko
-  node.specialArgs.self = self;
-  imports = [
-    test
-    ./container-driver/module.nix
-  ];
-})).config.result
+      # we have not private networking so far
+      networking.interfaces = lib.mkForce { };
+      #networking.primaryIPAddress = lib.mkForce null;
+      systemd.services.backdoor.enable = false;
+    };
+    # to accept external dependencies such as disko
+    node.specialArgs.self = self;
+    imports = [
+      test
+      ./container-driver/module.nix
+    ];
+  }
+)).config.result

checks/lib/test-base.nix

@@ -1,8 +1,5 @@
 test:
-{ pkgs
-, self
-, ...
-}:
+{ pkgs, self, ... }:
 let
   inherit (pkgs) lib;
   nixos-lib = import (pkgs.path + "/nixos/lib") { };

checks/matrix-synapse/default.nix

@@ -0,0 +1,37 @@
+(import ../lib/container-test.nix) (
+  { pkgs, ... }:
+  {
+    name = "matrix-synapse";
+
+    nodes.machine =
+      { self, lib, ... }:
+      {
+        imports = [
+          self.clanModules.matrix-synapse
+          self.nixosModules.clanCore
+          {
+            clanCore.machineName = "machine";
+            clanCore.clanDir = ./.;
+            clan.matrix-synapse = {
+              enable = true;
+              domain = "clan.test";
+            };
+          }
+          {
+            # secret override
+            clanCore.facts.services.matrix-synapse.secret.synapse-registration_shared_secret.path = "${./synapse-registration_shared_secret}";
+            services.nginx.virtualHosts."matrix.clan.test" = {
+              enableACME = lib.mkForce false;
+              forceSSL = lib.mkForce false;
+            };
+          }
+        ];
+      };
+    testScript = ''
+      start_all()
+      machine.wait_for_unit("matrix-synapse")
+      machine.succeed("${pkgs.netcat}/bin/nc -z -v ::1 8008")
+      machine.succeed("${pkgs.curl}/bin/curl -Ssf -L http://localhost/_matrix/static/ -H 'Host: matrix.clan.test'")
+    '';
+  }
+)

checks/matrix-synapse/synapse-registration_shared_secret

@@ -0,0 +1 @@
+registration_shared_secret: supersecret

checks/meshnamed/default.nix

@@ -1,21 +0,0 @@
-(import ../lib/container-test.nix) ({ pkgs, ... }: {
-  name = "meshnamed";
-
-  nodes.machine = { self, ... }: {
-    imports = [
-      self.nixosModules.clanCore
-      {
-        clanCore.machineName = "machine";
-        clan.networking.meshnamed.networks.vpn.subnet = "fd43:7def:4b50:28d0:4e99:9347:3035:17ef/88";
-        clanCore.clanDir = ./.;
-      }
-    ];
-  };
-  testScript = ''
-    start_all()
-    machine.wait_for_unit("meshnamed")
-    out = machine.succeed("${pkgs.dnsutils}/bin/dig AAAA foo.7vbx332lkaunatuzsndtanix54.vpn @meshnamed +short")
-    print(out)
-    assert out.strip() == "fd43:7def:4b50:28d0:4e99:9347:3035:17ef"
-  '';
-})

checks/schemas.nix

@@ -1,35 +0,0 @@
-{ self, runCommand, check-jsonschema, pkgs, lib, ... }:
-let
-  clanModules.clanCore = self.nixosModules.clanCore;
-
-  baseModule = {
-    imports =
-      (import (pkgs.path + "/nixos/modules/module-list.nix"))
-      ++ [{
-        nixpkgs.hostPlatform = "x86_64-linux";
-        clanCore.clanName = "dummy";
-      }];
-  };
-
-  optionsFromModule = module:
-    let
-      evaled = lib.evalModules {
-        modules = [ module baseModule ];
-      };
-    in
-    evaled.options.clan;
-
-  clanModuleSchemas = lib.mapAttrs (_: module: self.lib.jsonschema.parseOptions (optionsFromModule module)) clanModules;
-
-  mkTest = name: schema: runCommand "schema-${name}" { } ''
-    ${check-jsonschema}/bin/check-jsonschema \
-      --check-metaschema ${builtins.toFile "schema-${name}" (builtins.toJSON schema)}
-      touch $out
-  '';
-in
-lib.mapAttrs'
-  (name: schema: {
-    name = "schema-${name}";
-    value = mkTest name schema;
-  })
-  clanModuleSchemas

checks/secrets/default.nix

@@ -1,19 +1,20 @@
 (import ../lib/test-base.nix) {
   name = "secrets";
 
-  nodes.machine = { self, config, ... }: {
-    imports = [
-      (self.nixosModules.clanCore)
-    ];
-    environment.etc."secret".source = config.sops.secrets.secret.path;
-    environment.etc."group-secret".source = config.sops.secrets.group-secret.path;
-    sops.age.keyFile = ./key.age;
+  nodes.machine =
+    { self, config, ... }:
+    {
+      environment.etc."privkey.age".source = ./key.age;
+      imports = [ (self.nixosModules.clanCore) ];
+      environment.etc."secret".source = config.sops.secrets.secret.path;
+      environment.etc."group-secret".source = config.sops.secrets.group-secret.path;
+      sops.age.keyFile = "/etc/privkey.age";
 
-    clanCore.clanDir = "${./.}";
-    clanCore.machineName = "machine";
+      clanCore.clanDir = "${./.}";
+      clanCore.machineName = "machine";
 
-    networking.hostName = "machine";
-  };
+      networking.hostName = "machine";
+    };
   testScript = ''
     machine.succeed("cat /etc/secret >&2")
     machine.succeed("cat /etc/group-secret >&2")

checks/syncthing/default.nix

@@ -19,7 +19,7 @@
               "syncthing.key".source = ./introducer/introducer_test_key;
               "syncthing.api".source = ./introducer/introducer_test_api;
             };
-            clanCore.secrets.syncthing.secrets."syncthing.api".path = "/etc/syncthing.api";
+            clanCore.facts.services.syncthing.secret."syncthing.api".path = "/etc/syncthing.api";
             services.syncthing.cert = "/etc/syncthing.pam";
             services.syncthing.key = "/etc/syncthing.key";
             # Doesn't test zerotier!

checks/wayland-proxy-virtwl/default.nix

@@ -1,25 +1,35 @@
-import ../lib/test-base.nix ({ config, pkgs, lib, ... }: {
-  name = "wayland-proxy-virtwl";
+import ../lib/test-base.nix (
+  {
+    config,
+    pkgs,
+    lib,
+    ...
+  }:
+  {
+    name = "wayland-proxy-virtwl";
 
-  nodes.machine = { self, ... }: {
-    imports = [
-      self.nixosModules.clanCore
+    nodes.machine =
+      { self, ... }:
       {
-        clanCore.machineName = "machine";
-        clanCore.clanDir = ./.;
-      }
-    ];
-    services.wayland-proxy-virtwl.enable = true;
+        imports = [
+          self.nixosModules.clanCore
+          {
+            clanCore.machineName = "machine";
+            clanCore.clanDir = ./.;
+          }
+        ];
+        services.wayland-proxy-virtwl.enable = true;
 
-    virtualisation.qemu.options = [
-      "-vga none -device virtio-gpu-rutabaga,cross-domain=on,hostmem=4G,wsi=headless"
-    ];
+        virtualisation.qemu.options = [
+          "-vga none -device virtio-gpu-rutabaga,cross-domain=on,hostmem=4G,wsi=headless"
+        ];
 
-    virtualisation.qemu.package = lib.mkForce pkgs.qemu_kvm;
-  };
-  testScript = ''
-    start_all()
-    # use machinectl
-    machine.succeed("machinectl shell .host ${config.nodes.machine.systemd.package}/bin/systemctl --user start wayland-proxy-virtwl >&2")
-  '';
-})
+        virtualisation.qemu.package = lib.mkForce pkgs.qemu_kvm;
+      };
+    testScript = ''
+      start_all()
+      # use machinectl
+      machine.succeed("machinectl shell .host ${config.nodes.machine.systemd.package}/bin/systemctl --user start wayland-proxy-virtwl >&2")
+    '';
+  }
+)

checks/zt-tcp-relay/default.nix

@@ -1,20 +1,25 @@
-(import ../lib/container-test.nix) ({ pkgs, ... }: {
-  name = "zt-tcp-relay";
+(import ../lib/container-test.nix) (
+  { pkgs, ... }:
+  {
+    name = "zt-tcp-relay";
 
-  nodes.machine = { self, ... }: {
-    imports = [
-      self.nixosModules.clanCore
-      self.clanModules.zt-tcp-relay
+    nodes.machine =
+      { self, ... }:
       {
-        clanCore.machineName = "machine";
-        clanCore.clanDir = ./.;
-      }
-    ];
-  };
-  testScript = ''
-    start_all()
-    machine.wait_for_unit("zt-tcp-relay.service")
-    out = machine.succeed("${pkgs.netcat}/bin/nc -z -v localhost 4443")
-    print(out)
-  '';
-})
+        imports = [
+          self.nixosModules.clanCore
+          self.clanModules.zt-tcp-relay
+          {
+            clanCore.machineName = "machine";
+            clanCore.clanDir = ./.;
+          }
+        ];
+      };
+    testScript = ''
+      start_all()
+      machine.wait_for_unit("zt-tcp-relay.service")
+      out = machine.succeed("${pkgs.netcat}/bin/nc -z -v localhost 4443")
+      print(out)
+    '';
+  }
+)

clanModules/borgbackup.nix

@@ -1,94 +0,0 @@
-{ config, lib, pkgs, ... }:
-let
-  cfg = config.clan.borgbackup;
-in
-{
-  options.clan.borgbackup = {
-    enable = lib.mkEnableOption "backups with borgbackup";
-    destinations = lib.mkOption {
-      type = lib.types.attrsOf (lib.types.submodule ({ name, ... }: {
-        options = {
-          name = lib.mkOption {
-            type = lib.types.str;
-            default = name;
-            description = "the name of the backup job";
-          };
-          repo = lib.mkOption {
-            type = lib.types.str;
-            description = "the borgbackup repository to backup to";
-          };
-          rsh = lib.mkOption {
-            type = lib.types.str;
-            default = "ssh -i ${config.clanCore.secrets.borgbackup.secrets."borgbackup.ssh".path} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null";
-            description = "the rsh to use for the backup";
-          };
-
-        };
-      }));
-      description = ''
-        destinations where the machine should be backuped to
-      '';
-    };
-  };
-  config = lib.mkIf cfg.enable {
-    services.borgbackup.jobs = lib.mapAttrs
-      (_: dest: {
-        paths = lib.flatten (map (state: state.folders) (lib.attrValues config.clanCore.state));
-        exclude = [ "*.pyc" ];
-        repo = dest.repo;
-        environment.BORG_RSH = dest.rsh;
-        compression = "auto,zstd";
-        startAt = "*-*-* 01:00:00";
-        persistentTimer = true;
-        preHook = ''
-          set -x
-        '';
-
-        encryption = {
-          mode = "repokey";
-          passCommand = "cat ${config.clanCore.secrets.borgbackup.secrets."borgbackup.repokey".path}";
-        };
-
-        prune.keep = {
-          within = "1d"; # Keep all archives from the last day
-          daily = 7;
-          weekly = 4;
-          monthly = 0;
-        };
-      })
-      cfg.destinations;
-
-    clanCore.secrets.borgbackup = {
-      facts."borgbackup.ssh.pub" = { };
-      secrets."borgbackup.ssh" = { };
-      secrets."borgbackup.repokey" = { };
-      generator.path = [ pkgs.openssh pkgs.coreutils pkgs.xkcdpass ];
-      generator.script = ''
-        ssh-keygen -t ed25519 -N "" -f "$secrets"/borgbackup.ssh
-        mv "$secrets"/borgbackup.ssh.pub "$facts"/borgbackup.ssh.pub
-        xkcdpass -n 4 -d - > "$secrets"/borgbackup.repokey
-      '';
-    };
-
-    clanCore.backups.providers.borgbackup = {
-      # TODO list needs to run locally or on the remote machine
-      list = ''
-        # we need yes here to skip the changed url verification
-        ${lib.concatMapStringsSep "\n" (dest: ''yes y | borg-job-${dest.name} list --json | jq -r '. + {"job-name": "${dest.name}"}' '')
-          (lib.attrValues cfg.destinations)}
-      '';
-      create = ''
-        ${lib.concatMapStringsSep "\n" (dest: ''
-          systemctl start borgbackup-job-${dest.name}
-        '') (lib.attrValues cfg.destinations)}
-      '';
-
-      restore = ''
-        set -efu
-        cd /
-        IFS=';' read -ra FOLDER <<< "$FOLDERS"
-        yes y | borg-job-"$JOB" extract --list "$LOCATION"::"$ARCHIVE_ID" "''${FOLDER[@]}"
-      '';
-    };
-  };
-}

clanModules/borgbackup/README.md

@@ -0,0 +1,2 @@
+Efficient, deduplicating backup program with optional compression and secure encryption.
+---

clanModules/borgbackup/default.nix

@@ -0,0 +1,131 @@
+{
+  config,
+  lib,
+  pkgs,
+  ...
+}:
+let
+  cfg = config.clan.borgbackup;
+in
+{
+  options.clan.borgbackup.destinations = lib.mkOption {
+    type = lib.types.attrsOf (
+      lib.types.submodule (
+        { name, ... }:
+        {
+          options = {
+            name = lib.mkOption {
+              type = lib.types.strMatching "^[a-zA-Z0-9._-]+$";
+              default = name;
+              description = "the name of the backup job";
+            };
+            repo = lib.mkOption {
+              type = lib.types.str;
+              description = "the borgbackup repository to backup to";
+            };
+            rsh = lib.mkOption {
+              type = lib.types.str;
+              default = "ssh -i ${
+                config.clanCore.facts.services.borgbackup.secret."borgbackup.ssh".path
+              } -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null";
+              defaultText = "ssh -i \${config.clanCore.facts.services.borgbackup.secret.\"borgbackup.ssh\".path} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null";
+              description = "the rsh to use for the backup";
+            };
+          };
+        }
+      )
+    );
+    default = { };
+    description = ''
+      destinations where the machine should be backuped to
+    '';
+  };
+
+  imports = [
+    (lib.mkRemovedOptionModule [
+      "clan"
+      "borgbackup"
+      "enable"
+    ] "Just define clan.borgbackup.destinations to enable it")
+  ];
+
+  config = lib.mkIf (cfg.destinations != { }) {
+    services.borgbackup.jobs = lib.mapAttrs (_: dest: {
+      paths = lib.flatten (map (state: state.folders) (lib.attrValues config.clanCore.state));
+      exclude = [ "*.pyc" ];
+      repo = dest.repo;
+      environment.BORG_RSH = dest.rsh;
+      compression = "auto,zstd";
+      startAt = "*-*-* 01:00:00";
+      persistentTimer = true;
+      preHook = ''
+        set -x
+      '';
+
+      encryption = {
+        mode = "repokey";
+        passCommand = "cat ${config.clanCore.facts.services.borgbackup.secret."borgbackup.repokey".path}";
+      };
+
+      prune.keep = {
+        within = "1d"; # Keep all archives from the last day
+        daily = 7;
+        weekly = 4;
+        monthly = 0;
+      };
+    }) cfg.destinations;
+
+    clanCore.facts.services.borgbackup = {
+      public."borgbackup.ssh.pub" = { };
+      secret."borgbackup.ssh" = { };
+      secret."borgbackup.repokey" = { };
+      generator.path = [
+        pkgs.openssh
+        pkgs.coreutils
+        pkgs.xkcdpass
+      ];
+      generator.script = ''
+        ssh-keygen -t ed25519 -N "" -f "$secrets"/borgbackup.ssh
+        mv "$secrets"/borgbackup.ssh.pub "$facts"/borgbackup.ssh.pub
+        xkcdpass -n 4 -d - > "$secrets"/borgbackup.repokey
+      '';
+    };
+
+    environment.systemPackages = [
+      (pkgs.writeShellScriptBin "borgbackup-create" ''
+        set -efu -o pipefail
+        ${lib.concatMapStringsSep "\n" (dest: ''
+          systemctl start borgbackup-job-${dest.name}
+        '') (lib.attrValues cfg.destinations)}
+      '')
+      (pkgs.writeShellScriptBin "borgbackup-list" ''
+        set -efu
+        (${
+          lib.concatMapStringsSep "\n" (
+            dest:
+            # we need yes here to skip the changed url verification
+            ''yes y | borg-job-${dest.name} list --json | jq '[.archives[] | {"name": ("${dest.name}::${dest.repo}::" + .name)}]' ''
+          ) (lib.attrValues cfg.destinations)
+        }) | ${pkgs.jq}/bin/jq -s 'add'
+      '')
+      (pkgs.writeShellScriptBin "borgbackup-restore" ''
+        set -efux
+        cd /
+        IFS=';' read -ra FOLDER <<< "$FOLDERS"
+        job_name=$(echo "$NAME" | ${pkgs.gawk}/bin/awk -F'::' '{print $1}')
+        backup_name=''${NAME#"$job_name"::}
+        if ! command -v borg-job-"$job_name" &> /dev/null; then
+          echo "borg-job-$job_name not found: Backup name is invalid" >&2
+          exit 1
+        fi
+        yes y | borg-job-"$job_name" extract --list "$backup_name" "''${FOLDER[@]}"
+      '')
+    ];
+
+    clanCore.backups.providers.borgbackup = {
+      list = "borgbackup-list";
+      create = "borgbackup-create";
+      restore = "borgbackup-restore";
+    };
+  };
+}

clanModules/deltachat/README.md

@@ -0,0 +1,16 @@
+Email-based instant messaging for Desktop.
+---
+
+!!! warning "Under construction"
+
+!!! info
+    This module will automatically configure an email server on the machine for handling the e-mail messaging seamlessly.
+
+## Features
+
+- [x] **Email-based**: Uses any email account as its backend.
+- [x] **End-to-End Encryption**: Supports Autocrypt to automatically encrypt messages.
+- [x] **No Phone Number Required**: Uses your email address instead of a phone number.
+- [x] **Cross-Platform**: Available on desktop and mobile platforms.
+- [x] **Automatic Server Setup**: Includes your own DeltaChat server for enhanced control and privacy.
+- [ ] **Bake a cake**: This module cannot cake a bake.

clanModules/deltachat/default.nix

@@ -1,12 +1,11 @@
-{ config, pkgs, ... }: {
+{ config, pkgs, ... }:
+{
   networking.firewall.interfaces."zt+".allowedTCPPorts = [ 25 ]; # smtp with other hosts
   environment.systemPackages = [ pkgs.deltachat-desktop ];
 
   services.maddy =
     let
-      # FIXME move this to public setting
-      meshname = config.clanCore.secrets.zerotier.facts.zerotier-meshname.value or null;
-      domain = if meshname == null then "${config.clanCore.machineName}.local" else "${meshname}.vpn";
+      domain = "${config.clanCore.machineName}.local";
     in
     {
       enable = true;
@@ -136,9 +135,7 @@
           storage &local_mailboxes
         }
       '';
-      ensureAccounts = [
-        "user@${domain}"
-      ];
+      ensureAccounts = [ "user@${domain}" ];
       ensureCredentials = {
         "user@${domain}".passwordFile = pkgs.writeText "dummy" "foobar";
       };

clanModules/disk-layouts/README.md

@@ -0,0 +1,2 @@
+Automatically format a disk drive on clan installation
+---

clanModules/disk-layouts/default.nix

@@ -0,0 +1,48 @@
+{ config, lib, ... }:
+{
+  options.clan.disk-layouts.singleDiskExt4 = {
+    device = lib.mkOption {
+      type = lib.types.str;
+      example = "/dev/disk/by-id/ata-Samsung_SSD_850_EVO_250GB_S21PNXAGB12345";
+    };
+  };
+  config = {
+    boot.loader.grub.efiSupport = lib.mkDefault true;
+    boot.loader.grub.efiInstallAsRemovable = lib.mkDefault true;
+    disko.devices = {
+      disk = {
+        main = {
+          type = "disk";
+          device = config.clan.disk-layouts.singleDiskExt4.device;
+          content = {
+            type = "gpt";
+            partitions = {
+              boot = {
+                size = "1M";
+                type = "EF02"; # for grub MBR
+                priority = 1;
+              };
+              ESP = {
+                size = "512M";
+                type = "EF00";
+                content = {
+                  type = "filesystem";
+                  format = "vfat";
+                  mountpoint = "/boot";
+                };
+              };
+              root = {
+                size = "100%";
+                content = {
+                  type = "filesystem";
+                  format = "ext4";
+                  mountpoint = "/";
+                };
+              };
+            };
+          };
+        };
+      };
+    };
+  };
+}

clanModules/ergochat/README.md

@@ -0,0 +1,2 @@
+A modern IRC server
+---

clanModules/ergochat/default.nix

@@ -0,0 +1,14 @@
+_: {
+  services.ergochat = {
+    enable = true;
+
+    settings = {
+      datastore = {
+        autoupgrade = true;
+        path = "/var/lib/ergo/ircd.db";
+      };
+    };
+  };
+
+  clanCore.state.ergochat.folders = [ "/var/lib/ergo" ];
+}

clanModules/flake-module.nix

@@ -1,19 +1,26 @@
-{ inputs, ... }: {
+{ ... }:
+{
   flake.clanModules = {
-    diskLayouts = {
-      imports = [
-        ./diskLayouts.nix
-        inputs.disko.nixosModules.default
-      ];
+    disk-layouts = {
+      imports = [ ./disk-layouts ];
     };
-    borgbackup = ./borgbackup.nix;
-    deltachat = ./deltachat.nix;
-    moonlight = ./moonlight.nix;
-    sunshine = ./sunshine.nix;
-    syncthing = ./syncthing.nix;
-    xfce = ./xfce.nix;
-    zt-tcp-relay = ./zt-tcp-relay.nix;
-    localsend = ./localsend.nix;
-    waypipe = ./waypipe.nix;
+    borgbackup = ./borgbackup;
+    deltachat = ./deltachat;
+    ergochat = ./ergochat;
+    localbackup = ./localbackup;
+    localsend = ./localsend;
+    matrix-synapse = ./matrix-synapse;
+    moonlight = ./moonlight;
+    root-password = ./root-password;
+    sshd = ./sshd;
+    sunshine = ./sunshine;
+    static-hosts = ./static-hosts;
+    syncthing = ./syncthing;
+    thelounge = ./thelounge;
+    trusted-nix-caches = ./trusted-nix-caches;
+    user-password = ./user-password;
+    xfce = ./xfce;
+    zerotier-static-peers = ./zerotier-static-peers;
+    zt-tcp-relay = ./zt-tcp-relay;
   };
 }

clanModules/localbackup/README.md

@@ -0,0 +1,2 @@
+Automatically backups current machine to local directory.
+---

clanModules/localbackup/default.nix

@@ -0,0 +1,223 @@
+{
+  config,
+  lib,
+  pkgs,
+  ...
+}:
+let
+  cfg = config.clan.localbackup;
+  rsnapshotConfig = target: states: ''
+    config_version	1.2
+    snapshot_root	${target.directory}
+    sync_first	1
+    cmd_cp	${pkgs.coreutils}/bin/cp
+    cmd_rm	${pkgs.coreutils}/bin/rm
+    cmd_rsync	${pkgs.rsync}/bin/rsync
+    cmd_ssh	${pkgs.openssh}/bin/ssh
+    cmd_logger	${pkgs.inetutils}/bin/logger
+    cmd_du	${pkgs.coreutils}/bin/du
+    cmd_rsnapshot_diff	${pkgs.rsnapshot}/bin/rsnapshot-diff
+    ${lib.optionalString (target.preBackupHook != null) ''
+      cmd_preexec	${pkgs.writeShellScript "preexec.sh" ''
+        set -efu -o pipefail
+        ${target.preBackupHook}
+      ''}
+    ''}
+
+    ${lib.optionalString (target.postBackupHook != null) ''
+      cmd_postexec	${pkgs.writeShellScript "postexec.sh" ''
+        set -efu -o pipefail
+        ${target.postBackupHook}
+      ''}
+    ''}
+    retain	snapshot	${builtins.toString config.clan.localbackup.snapshots}
+    ${lib.concatMapStringsSep "\n" (state: ''
+      ${lib.concatMapStringsSep "\n" (folder: ''
+        backup	${folder}	${config.networking.hostName}/
+      '') state.folders}
+    '') states}
+  '';
+in
+{
+  options.clan.localbackup = {
+    targets = lib.mkOption {
+      type = lib.types.attrsOf (
+        lib.types.submodule (
+          { name, ... }:
+          {
+            options = {
+              name = lib.mkOption {
+                type = lib.types.strMatching "^[a-zA-Z0-9._-]+$";
+                default = name;
+                description = "the name of the backup job";
+              };
+              directory = lib.mkOption {
+                type = lib.types.str;
+                description = "the directory to backup";
+              };
+              mountpoint = lib.mkOption {
+                type = lib.types.nullOr lib.types.str;
+                default = null;
+                description = "mountpoint of the directory to backup. If set, the directory will be mounted before the backup and unmounted afterwards";
+              };
+              preMountHook = lib.mkOption {
+                type = lib.types.nullOr lib.types.lines;
+                default = null;
+                description = "Shell commands to run before the directory is mounted";
+              };
+              postMountHook = lib.mkOption {
+                type = lib.types.nullOr lib.types.lines;
+                default = null;
+                description = "Shell commands to run after the directory is mounted";
+              };
+              preUnmountHook = lib.mkOption {
+                type = lib.types.nullOr lib.types.lines;
+                default = null;
+                description = "Shell commands to run before the directory is unmounted";
+              };
+              postUnmountHook = lib.mkOption {
+                type = lib.types.nullOr lib.types.lines;
+                default = null;
+                description = "Shell commands to run after the directory is unmounted";
+              };
+              preBackupHook = lib.mkOption {
+                type = lib.types.nullOr lib.types.lines;
+                default = null;
+                description = "Shell commands to run before the backup";
+              };
+              postBackupHook = lib.mkOption {
+                type = lib.types.nullOr lib.types.lines;
+                default = null;
+                description = "Shell commands to run after the backup";
+              };
+            };
+          }
+        )
+      );
+      default = { };
+      description = "List of directories where backups are stored";
+    };
+
+    snapshots = lib.mkOption {
+      type = lib.types.int;
+      default = 20;
+      description = "Number of snapshots to keep";
+    };
+  };
+
+  config =
+    let
+      mountHook = target: ''
+        if [[ -x /run/current-system/sw/bin/localbackup-mount-${target.name} ]]; then
+          /run/current-system/sw/bin/localbackup-mount-${target.name}
+        fi
+        if [[ -x /run/current-system/sw/bin/localbackup-unmount-${target.name} ]]; then
+          trap "/run/current-system/sw/bin/localbackup-unmount-${target.name}" EXIT
+        fi
+      '';
+    in
+    lib.mkIf (cfg.targets != { }) {
+      environment.systemPackages =
+        [
+          (pkgs.writeShellScriptBin "localbackup-create" ''
+            set -efu -o pipefail
+            export PATH=${
+              lib.makeBinPath [
+                pkgs.rsnapshot
+                pkgs.coreutils
+                pkgs.util-linux
+              ]
+            }
+            ${lib.concatMapStringsSep "\n" (target: ''
+              (
+                ${mountHook target}
+                echo "Creating backup '${target.name}'"
+                rsnapshot -c "${pkgs.writeText "rsnapshot.conf" (rsnapshotConfig target (lib.attrValues config.clanCore.state))}" sync
+                rsnapshot -c "${pkgs.writeText "rsnapshot.conf" (rsnapshotConfig target (lib.attrValues config.clanCore.state))}" snapshot
+              )
+            '') (builtins.attrValues cfg.targets)}
+          '')
+          (pkgs.writeShellScriptBin "localbackup-list" ''
+            set -efu -o pipefail
+            export PATH=${
+              lib.makeBinPath [
+                pkgs.jq
+                pkgs.findutils
+                pkgs.coreutils
+                pkgs.util-linux
+              ]
+            }
+            (${
+              lib.concatMapStringsSep "\n" (target: ''
+                (
+                  ${mountHook target}
+                  find ${lib.escapeShellArg target.directory} -mindepth 1 -maxdepth 1 -name "snapshot.*" -print0 -type d \
+                    | jq -Rs 'split("\u0000") | .[] | select(. != "") | { "name": ("${target.name}::" + .)}'
+                )
+              '') (builtins.attrValues cfg.targets)
+            }) | jq -s .
+          '')
+          (pkgs.writeShellScriptBin "localbackup-restore" ''
+            set -efu -o pipefail
+            export PATH=${
+              lib.makeBinPath [
+                pkgs.rsync
+                pkgs.coreutils
+                pkgs.util-linux
+                pkgs.gawk
+              ]
+            }
+            name=$(awk -F'::' '{print $1}' <<< $NAME)
+            backupname=''${NAME#$name::}
+
+            if command -v localbackup-mount-$name; then
+              localbackup-mount-$name
+            fi
+            if command -v localbackup-unmount-$name; then
+              trap "localbackup-unmount-$name" EXIT
+            fi
+
+            if [[ ! -d $backupname ]]; then
+              echo "No backup found $backupname"
+              exit 1
+            fi
+
+            IFS=';' read -ra FOLDER <<< "$FOLDERS"
+            for folder in "''${FOLDER[@]}"; do
+              rsync -a "$backupname/${config.networking.hostName}$folder/" "$folder"
+            done
+          '')
+        ]
+        ++ (lib.mapAttrsToList (
+          name: target:
+          pkgs.writeShellScriptBin ("localbackup-mount-" + name) ''
+            set -efu -o pipefail
+            ${lib.optionalString (target.preMountHook != null) target.preMountHook}
+            ${lib.optionalString (target.mountpoint != null) ''
+              if ! ${pkgs.util-linux}/bin/mountpoint -q ${lib.escapeShellArg target.mountpoint}; then
+                ${pkgs.util-linux}/bin/mount -o X-mount.mkdir ${lib.escapeShellArg target.mountpoint}
+              fi
+            ''}
+            ${lib.optionalString (target.postMountHook != null) target.postMountHook}
+          ''
+        ) cfg.targets)
+        ++ lib.mapAttrsToList (
+          name: target:
+          pkgs.writeShellScriptBin ("localbackup-unmount-" + name) ''
+            set -efu -o pipefail
+            ${lib.optionalString (target.preUnmountHook != null) target.preUnmountHook}
+            ${lib.optionalString (
+              target.mountpoint != null
+            ) "${pkgs.util-linux}/bin/umount ${lib.escapeShellArg target.mountpoint}"}
+            ${lib.optionalString (target.postUnmountHook != null) target.postUnmountHook}
+          ''
+        ) cfg.targets;
+
+      clanCore.backups.providers.localbackup = {
+        # TODO list needs to run locally or on the remote machine
+        list = "localbackup-list";
+        create = "localbackup-create";
+        restore = "localbackup-restore";
+      };
+    };
+}

clanModules/localsend.nix

@@ -1,43 +0,0 @@
-{ config
-, pkgs
-, lib
-, ...
-}:
-{
-  # Integration can be improved, if the following issues get implemented:
-  # - cli frontend: https://github.com/localsend/localsend/issues/11
-  # - ipv6 support: https://github.com/localsend/localsend/issues/549
-  options.clan.localsend = {
-    enable = lib.mkEnableOption (lib.mdDoc "enable the localsend module");
-    defaultLocation = lib.mkOption {
-      type = lib.types.str;
-      description = "The default download location";
-    };
-    package = lib.mkPackageOption pkgs "localsend" { };
-  };
-
-  imports =
-    if config.clan.localsend.enable then
-      [
-        {
-          clanCore.state.localsend.folders = [
-            "/var/localsend"
-            config.clan.localsend.defaultLocation
-          ];
-          environment.systemPackages = [ config.clan.localsend.package ];
-
-          networking.firewall.interfaces."zt+".allowedTCPPorts = [ 53317 ];
-          networking.firewall.interfaces."zt+".allowedUDPPorts = [ 53317 ];
-
-          #TODO: This is currently needed because there is no ipv6 multicasting support yet
-          #
-          systemd.network.networks."09-zerotier" = {
-            networkConfig = {
-              Address = "192.168.56.2/24";
-            };
-          };
-        }
-      ]
-    else
-      [ ];
-}

clanModules/localsend/README.md

@@ -0,0 +1,2 @@
+Securely sharing files and messages over a local network without internet connectivity.
+---

clanModules/localsend/default.nix

@@ -0,0 +1,38 @@
+{
+  config,
+  pkgs,
+  lib,
+  ...
+}:
+{
+  # Integration can be improved, if the following issues get implemented:
+  # - cli frontend: https://github.com/localsend/localsend/issues/11
+  # - ipv6 support: https://github.com/localsend/localsend/issues/549
+  options.clan.localsend = {
+    enable = lib.mkEnableOption (lib.mdDoc "enable the localsend module");
+    defaultLocation = lib.mkOption {
+      type = lib.types.str;
+      description = "The default download location";
+    };
+    package = lib.mkPackageOption pkgs "localsend" { };
+  };
+
+  config = lib.mkIf config.clan.localsend.enable {
+    clanCore.state.localsend.folders = [
+      "/var/localsend"
+      config.clan.localsend.defaultLocation
+    ];
+    environment.systemPackages = [ config.clan.localsend.package ];
+
+    networking.firewall.interfaces."zt+".allowedTCPPorts = [ 53317 ];
+    networking.firewall.interfaces."zt+".allowedUDPPorts = [ 53317 ];
+
+    #TODO: This is currently needed because there is no ipv6 multicasting support yet
+    #
+    systemd.network.networks."09-zerotier" = {
+      networkConfig = {
+        Address = "192.168.56.2/24";
+      };
+    };
+  };
+}

clanModules/matrix-synapse/README.md

@@ -0,0 +1,2 @@
+A federated messaging server with end-to-end encryption.
+---

clanModules/matrix-synapse/default.nix

@@ -0,0 +1,127 @@
+{
+  config,
+  lib,
+  pkgs,
+  ...
+}:
+let
+  cfg = config.clan.matrix-synapse;
+in
+{
+  options.clan.matrix-synapse = {
+    enable = lib.mkEnableOption "Enable matrix-synapse";
+    domain = lib.mkOption {
+      type = lib.types.str;
+      description = "The domain name of the matrix server";
+    };
+  };
+  config = lib.mkIf cfg.enable {
+    services.matrix-synapse = {
+      enable = true;
+      settings = {
+        server_name = cfg.domain;
+        database = {
+          args.user = "matrix-synapse";
+          args.database = "matrix-synapse";
+          name = "psycopg2";
+        };
+        turn_uris = [
+          "turn:turn.matrix.org?transport=udp"
+          "turn:turn.matrix.org?transport=tcp"
+        ];
+        listeners = [
+          {
+            port = 8008;
+            bind_addresses = [ "::1" ];
+            type = "http";
+            tls = false;
+            x_forwarded = true;
+            resources = [
+              {
+                names = [ "client" ];
+                compress = true;
+              }
+              {
+                names = [ "federation" ];
+                compress = false;
+              }
+            ];
+          }
+        ];
+      };
+      extraConfigFiles = [ "/var/lib/matrix-synapse/registration_shared_secret.yaml" ];
+    };
+    systemd.services.matrix-synapse.serviceConfig.ExecStartPre = [
+      "+${pkgs.writeScript "copy_registration_shared_secret" ''
+        #!/bin/sh
+        cp ${config.clanCore.facts.services.matrix-synapse.secret.synapse-registration_shared_secret.path} /var/lib/matrix-synapse/registration_shared_secret.yaml
+        chown matrix-synapse:matrix-synapse /var/lib/matrix-synapse/registration_shared_secret.yaml
+        chmod 600 /var/lib/matrix-synapse/registration_shared_secret.yaml
+      ''}"
+    ];
+
+    clanCore.facts.services."matrix-synapse" = {
+      secret."synapse-registration_shared_secret" = { };
+      generator.path = with pkgs; [
+        coreutils
+        pwgen
+      ];
+      generator.script = ''
+        echo "registration_shared_secret: $(pwgen -s 32 1)" > "$secrets"/synapse-registration_shared_secret
+      '';
+    };
+
+    services.postgresql.enable = true;
+    # we need to use both ensusureDatabases and initialScript, because the former runs everytime but with the wrong collation
+    services.postgresql = {
+      ensureDatabases = [ "matrix-synapse" ];
+      ensureUsers = [
+        {
+          name = "matrix-synapse";
+          ensureDBOwnership = true;
+        }
+      ];
+      initialScript = pkgs.writeText "synapse-init.sql" ''
+        CREATE DATABASE "matrix-synapse"
+          TEMPLATE template0
+          LC_COLLATE = "C"
+          LC_CTYPE = "C";
+      '';
+    };
+    services.nginx = {
+      enable = true;
+      virtualHosts = {
+        ${cfg.domain} = {
+          locations."= /.well-known/matrix/server".extraConfig = ''
+            add_header Content-Type application/json;
+            return 200 '${builtins.toJSON { "m.server" = "matrix.${cfg.domain}:443"; }}';
+          '';
+          locations."= /.well-known/matrix/client".extraConfig = ''
+            add_header Content-Type application/json;
+            add_header Access-Control-Allow-Origin *;
+            return 200 '${
+              builtins.toJSON {
+                "m.homeserver" = {
+                  "base_url" = "https://matrix.${cfg.domain}";
+                };
+                "m.identity_server" = {
+                  "base_url" = "https://vector.im";
+                };
+              }
+            }';
+          '';
+        };
+        "matrix.${cfg.domain}" = {
+          forceSSL = true;
+          enableACME = true;
+          locations."/_matrix" = {
+            proxyPass = "http://localhost:8008";
+          };
+          locations."/test".extraConfig = ''
+            return 200 "Hello, world!";
+          '';
+        };
+      };
+    };
+  };
+}

clanModules/moonlight.nix

@@ -1,4 +0,0 @@
-{ pkgs, ... }: {
-  hardware.opengl.enable = true;
-  environment.systemPackages = [ pkgs.moonlight-qt ];
-}

clanModules/moonlight/README.md

@@ -0,0 +1,2 @@
+A desktop streaming client optimized for remote gaming and synchronized movie viewing.
+---

clanModules/moonlight/default.nix

@@ -0,0 +1,86 @@
+{ pkgs, config, ... }:
+let
+  ms-accept = pkgs.callPackage ../pkgs/moonlight-sunshine-accept { };
+  defaultPort = 48011;
+in
+{
+  hardware.opengl.enable = true;
+  environment.systemPackages = [
+    pkgs.moonlight-qt
+    ms-accept
+  ];
+
+  systemd.tmpfiles.rules = [
+    "d '/var/lib/moonlight' 0770 'user' 'users' - -"
+    "C '/var/lib/moonlight/moonlight.cert' 0644 'user' 'users' - ${
+      config.clanCore.facts.services.moonlight.secret."moonlight.cert".path or ""
+    }"
+    "C '/var/lib/moonlight/moonlight.key' 0644 'user' 'users' - ${
+      config.clanCore.facts.services.moonlight.secret."moonlight.key".path or ""
+    }"
+  ];
+
+  systemd.user.services.init-moonlight = {
+    enable = false;
+    description = "Initializes moonlight";
+    wantedBy = [ "graphical-session.target" ];
+    script = ''
+      ${ms-accept}/bin/moonlight-sunshine-accept moonlight init-config --key /var/lib/moonlight/moonlight.key --cert /var/lib/moonlight/moonlight.cert
+    '';
+    serviceConfig = {
+      user = "user";
+      Type = "oneshot";
+      WorkingDirectory = "/home/user/";
+      RunTimeDirectory = "moonlight";
+      TimeoutSec = "infinity";
+      Restart = "on-failure";
+      RemainAfterExit = true;
+      ReadOnlyPaths = [
+        "/var/lib/moonlight/moonlight.key"
+        "/var/lib/moonlight/moonlight.cert"
+      ];
+    };
+  };
+
+  systemd.user.services.moonlight-join = {
+    description = "Join sunshine hosts";
+    script = ''${ms-accept}/bin/moonlight-sunshine-accept moonlight join --port ${builtins.toString defaultPort} --cert '${
+      config.clanCore.facts.services.moonlight.public."moonlight.cert".value or ""
+    }' --host fd2e:25da:6035:c98f:cd99:93e0:b9b8:9ca1'';
+    serviceConfig = {
+      Type = "oneshot";
+      TimeoutSec = "infinity";
+      Restart = "on-failure";
+      ReadOnlyPaths = [
+        "/var/lib/moonlight/moonlight.key"
+        "/var/lib/moonlight/moonlight.cert"
+      ];
+    };
+  };
+  systemd.user.timers.moonlight-join = {
+    description = "Join sunshine hosts";
+    wantedBy = [ "timers.target" ];
+    timerConfig = {
+      OnUnitActiveSec = "5min";
+      OnBootSec = "0min";
+      Persistent = true;
+      Unit = "moonlight-join.service";
+    };
+  };
+
+  clanCore.facts.services.moonlight = {
+    secret."moonlight.key" = { };
+    secret."moonlight.cert" = { };
+    public."moonlight.cert" = { };
+    generator.path = [
+      pkgs.coreutils
+      ms-accept
+    ];
+    generator.script = ''
+      moonlight-sunshine-accept moonlight init
+      mv credentials/cakey.pem "$secrets"/moonlight.key
+      cp credentials/cacert.pem "$secrets"/moonlight.cert
+      mv credentials/cacert.pem "$facts"/moonlight.cert
+    '';
+  };
+}

clanModules/root-password/README.md

@@ -0,0 +1,11 @@
+Automatically generates and configures a password for the root user.
+---
+
+After the system was installed/deployed the following command can be used to display the root-password:
+
+```bash
+clan secrets get {machine_name}-password
+```
+
+
+See also: [Facts / Secrets](../../getting-started/secrets.md)

clanModules/root-password/default.nix

@@ -0,0 +1,20 @@
+{ pkgs, config, ... }:
+{
+  users.mutableUsers = false;
+  users.users.root.hashedPasswordFile =
+    config.clanCore.facts.services.root-password.secret.password-hash.path;
+  sops.secrets."${config.clanCore.machineName}-password-hash".neededForUsers = true;
+  clanCore.facts.services.root-password = {
+    secret.password = { };
+    secret.password-hash = { };
+    generator.path = with pkgs; [
+      coreutils
+      xkcdpass
+      mkpasswd
+    ];
+    generator.script = ''
+      xkcdpass --numwords 3 --delimiter - --count 1 > $secrets/password
+      cat $secrets/password | mkpasswd -s -m sha-512 > $secrets/password-hash
+    '';
+  };
+}

clanModules/sshd/README.md

@@ -0,0 +1,2 @@
+Enables secure remote access to the machine over ssh
+---

clanModules/sshd/default.nix

@@ -0,0 +1,25 @@
+{ config, pkgs, ... }:
+{
+  services.openssh.enable = true;
+  services.openssh.settings.PasswordAuthentication = false;
+
+  services.openssh.hostKeys = [
+    {
+      path = config.clanCore.facts.services.openssh.secret."ssh.id_ed25519".path;
+      type = "ed25519";
+    }
+  ];
+
+  clanCore.facts.services.openssh = {
+    secret."ssh.id_ed25519" = { };
+    public."ssh.id_ed25519.pub" = { };
+    generator.path = [
+      pkgs.coreutils
+      pkgs.openssh
+    ];
+    generator.script = ''
+      ssh-keygen -t ed25519 -N "" -f $secrets/ssh.id_ed25519
+      mv $secrets/ssh.id_ed25519.pub $facts/ssh.id_ed25519.pub
+    '';
+  };
+}

clanModules/static-hosts/README.md

@@ -0,0 +1,2 @@
+Statically configure the host names of machines based on their respective zerotier-ip.
+---

clanModules/static-hosts/default.nix

@@ -0,0 +1,30 @@
+{ lib, config, ... }:
+{
+  options.clan.static-hosts = {
+    excludeHosts = lib.mkOption {
+      type = lib.types.listOf lib.types.str;
+      default = [ config.clanCore.machineName ];
+      description = "Hosts that should be excluded";
+    };
+  };
+
+  config.networking.hosts =
+    let
+      clanDir = config.clanCore.clanDir;
+      machineDir = clanDir + "/machines/";
+      zerotierIpMachinePath = machines: machineDir + machines + "/facts/zerotier-ip";
+      machines = builtins.readDir machineDir;
+      filteredMachines = lib.filterAttrs (
+        name: _: !(lib.elem name config.clan.static-hosts.excludeHosts)
+      ) machines;
+    in
+    lib.filterAttrs (_: value: value != null) (
+      lib.mapAttrs' (
+        machine: _:
+        let
+          path = zerotierIpMachinePath machine;
+        in
+        if builtins.pathExists path then lib.nameValuePair (builtins.readFile path) [ machine ] else null
+      ) filteredMachines
+    );
+}

clanModules/sunshine.nix

@@ -1,109 +0,0 @@
-{ pkgs, config, ... }:
-{
-  networking.firewall = {
-    allowedTCPPorts = [
-      47984
-      47989
-      47990
-      48010
-    ];
-
-    allowedUDPPorts = [
-      47998
-      47999
-      48000
-      48002
-      48010
-    ];
-  };
-
-  networking.firewall.allowedTCPPortRanges = [
-    {
-      from = 47984;
-      to = 48010;
-    }
-  ];
-  networking.firewall.allowedUDPPortRanges = [
-    {
-      from = 47998;
-      to = 48010;
-    }
-  ];
-
-  environment.systemPackages = [
-    pkgs.sunshine
-    pkgs.avahi
-    # Convenience script, until we find a better UX
-    (pkgs.writers.writeDashBin "sun" ''
-      ${pkgs.sunshine}/bin/sunshine -1 ${
-        pkgs.writeText "sunshine.conf" ''
-          address_family = both
-        ''
-      } "$@"
-    '')
-    # Create a dummy account, for easier setup,
-    # don't use this account in actual production yet.
-    (pkgs.writers.writeDashBin "init-sun" ''
-      ${pkgs.sunshine}/bin/sunshine \
-      --creds "sun" "sun"
-    '')
-  ];
-
-  # Required to simulate input
-  boot.kernelModules = [ "uinput" ];
-  security.rtkit.enable = true;
-
-  # services.udev.extraRules = ''
-  #   KERNEL=="uinput", SUBSYSTEM=="misc", OPTIONS+="static_node=uinput", TAG+="uaccess"
-  # '';
-
-  services.udev.extraRules = ''
-    KERNEL=="uinput", GROUP="input", MODE="0660" OPTIONS+="static_node=uinput"
-  '';
-
-  security.wrappers.sunshine = {
-    owner = "root";
-    group = "root";
-    capabilities = "cap_sys_admin+p";
-    source = "${pkgs.sunshine}/bin/sunshine";
-  };
-
-  systemd.user.services.sunshine = {
-    description = "sunshine";
-    wantedBy = [ "graphical-session.target" ];
-    environment = {
-      DISPLAY = ":0";
-    };
-    serviceConfig = {
-      ExecStart = "${config.security.wrapperDir}/sunshine";
-    };
-  };
-
-  # xdg.configFile."sunshine/apps.json".text = builtins.toJSON {
-  #   env = "/run/current-system/sw/bin";
-  #   apps = [
-  #     {
-  #       name = "Steam";
-  #       output = "steam.txt";
-  #       detached = [
-  #         "${pkgs.util-linux}/bin/setsid ${pkgs.steam}/bin/steam steam://open/bigpicture"
-  #       ];
-  #       image-path = "steam.png";
-  #     }
-  #   ];
-  # };
-
-  services = {
-    avahi = {
-      enable = true;
-      reflector = true;
-      nssmdns = true;
-      publish = {
-        enable = true;
-        addresses = true;
-        userServices = true;
-        workstation = true;
-      };
-    };
-  };
-}

clanModules/sunshine/README.md

@@ -0,0 +1,2 @@
+A desktop streaming server optimized for remote gaming and synchronized movie viewing.
+---

clanModules/sunshine/default.nix

@@ -0,0 +1,207 @@
+{
+  pkgs,
+  config,
+  lib,
+  ...
+}:
+let
+  ms-accept = pkgs.callPackage ../pkgs/moonlight-sunshine-accept { };
+  sunshineConfiguration = pkgs.writeText "sunshine.conf" ''
+    address_family = both
+    channels = 5
+    pkey = /var/lib/sunshine/sunshine.key
+    cert = /var/lib/sunshine/sunshine.cert
+    file_state = /var/lib/sunshine/state.json
+    credentials_file = /var/lib/sunshine/credentials.json
+  '';
+  listenPort = 48011;
+in
+{
+  networking.firewall = {
+    allowedTCPPorts = [
+      47984
+      47989
+      47990
+      48010
+      48011
+    ];
+
+    allowedUDPPorts = [
+      47998
+      47999
+      48000
+      48002
+      48010
+    ];
+  };
+
+  networking.firewall.allowedTCPPortRanges = [
+    {
+      from = 47984;
+      to = 48010;
+    }
+  ];
+  networking.firewall.allowedUDPPortRanges = [
+    {
+      from = 47998;
+      to = 48010;
+    }
+  ];
+  networking.firewall.interfaces."zt+".allowedTCPPorts = [
+    47984
+    47989
+    47990
+    48010
+    listenPort
+  ];
+  networking.firewall.interfaces."zt+".allowedUDPPortRanges = [
+    {
+      from = 47998;
+      to = 48010;
+    }
+  ];
+
+  environment.systemPackages = [
+    ms-accept
+    pkgs.sunshine
+    pkgs.avahi
+    # Convenience script, until we find a better UX
+    (pkgs.writers.writeDashBin "sun" ''
+      ${pkgs.sunshine}/bin/sunshine -0 ${sunshineConfiguration} "$@"
+    '')
+    # Create a dummy account, for easier setup,
+    # don't use this account in actual production yet.
+    (pkgs.writers.writeDashBin "init-sun" ''
+      ${pkgs.sunshine}/bin/sunshine \
+      --creds "sunshine" "sunshine"
+    '')
+  ];
+
+  # Required to simulate input
+  boot.kernelModules = [ "uinput" ];
+
+  services.udev.extraRules = ''
+    KERNEL=="uinput", SUBSYSTEM=="misc", OPTIONS+="static_node=uinput", TAG+="uaccess"
+  '';
+
+  security = {
+    rtkit.enable = true;
+    wrappers.sunshine = {
+      owner = "root";
+      group = "root";
+      capabilities = "cap_sys_admin+p";
+      source = "${pkgs.sunshine}/bin/sunshine";
+    };
+  };
+
+  systemd.tmpfiles.rules = [
+    "d '/var/lib/sunshine' 0770 'user' 'users' - -"
+    "C '/var/lib/sunshine/sunshine.cert' 0644 'user' 'users' - ${
+      config.clanCore.facts.services.sunshine.secret."sunshine.cert".path or ""
+    }"
+    "C '/var/lib/sunshine/sunshine.key' 0644 'user' 'users' - ${
+      config.clanCore.facts.services.sunshine.secret."sunshine.key".path or ""
+    }"
+  ];
+
+  hardware.opengl.enable = true;
+
+  systemd.user.services.sunshine = {
+    enable = true;
+    description = "Sunshine self-hosted game stream host for Moonlight";
+    startLimitBurst = 5;
+    startLimitIntervalSec = 500;
+    script = "/run/current-system/sw/bin/env /run/wrappers/bin/sunshine ${sunshineConfiguration}";
+    serviceConfig = {
+      Restart = "on-failure";
+      RestartSec = "5s";
+      ReadWritePaths = [ "/var/lib/sunshine" ];
+      ReadOnlyPaths = [
+        (config.clanCore.facts.services.sunshine.secret."sunshine.key".path or "")
+        (config.clanCore.facts.services.sunshine.secret."sunshine.cert".path or "")
+      ];
+    };
+    wantedBy = [ "graphical-session.target" ];
+    partOf = [ "graphical-session.target" ];
+    wants = [ "graphical-session.target" ];
+    after = [
+      "sunshine-init-state.service"
+      "sunshine-init-credentials.service"
+    ];
+  };
+
+  systemd.user.services.sunshine-init-state = {
+    enable = true;
+    description = "Sunshine self-hosted game stream host for Moonlight";
+    startLimitBurst = 5;
+    startLimitIntervalSec = 500;
+    script = ''
+      ${ms-accept}/bin/moonlight-sunshine-accept sunshine init-state --uuid ${
+        config.clanCore.facts.services.sunshine.public.sunshine-uuid.value or null
+      } --state-file /var/lib/sunshine/state.json
+    '';
+    serviceConfig = {
+      Restart = "on-failure";
+      RestartSec = "5s";
+      Type = "oneshot";
+      ReadWritePaths = [ "/var/lib/sunshine" ];
+    };
+    wantedBy = [ "graphical-session.target" ];
+  };
+
+  systemd.user.services.sunshine-init-credentials = {
+    enable = true;
+    description = "Sunshine self-hosted game stream host for Moonlight";
+    startLimitBurst = 5;
+    startLimitIntervalSec = 500;
+    script = ''
+      ${lib.getExe pkgs.sunshine} ${sunshineConfiguration} --creds sunshine sunshine
+    '';
+    serviceConfig = {
+      Restart = "on-failure";
+      RestartSec = "5s";
+      Type = "oneshot";
+      ReadWritePaths = [ "/var/lib/sunshine" ];
+    };
+    wantedBy = [ "graphical-session.target" ];
+  };
+
+  systemd.user.services.sunshine-listener = {
+    enable = true;
+    description = "Sunshine self-hosted game stream host for Moonlight";
+    startLimitBurst = 5;
+    startLimitIntervalSec = 500;
+    script = ''
+      ${ms-accept}/bin/moonlight-sunshine-accept sunshine listen --port ${builtins.toString listenPort} --uuid ${
+        config.clanCore.facts.services.sunshine.public.sunshine-uuid.value or null
+      }  --state /var/lib/sunshine/state.json --cert '${
+        config.clanCore.facts.services.sunshine.public."sunshine.cert".value or null
+      }'
+    '';
+    serviceConfig = {
+      # );
+      Restart = "on-failure";
+      RestartSec = 5;
+      ReadWritePaths = [ "/var/lib/sunshine" ];
+    };
+    wantedBy = [ "graphical-session.target" ];
+  };
+
+  clanCore.facts.services.ergochat = {
+    secret."sunshine.key" = { };
+    secret."sunshine.cert" = { };
+    public."sunshine-uuid" = { };
+    public."sunshine.cert" = { };
+    generator.path = [
+      pkgs.coreutils
+      ms-accept
+    ];
+    generator.script = ''
+      moonlight-sunshine-accept sunshine init
+      mv credentials/cakey.pem "$secrets"/sunshine.key
+      cp credentials/cacert.pem "$secrets"/sunshine.cert
+      mv credentials/cacert.pem "$facts"/sunshine.cert
+      mv uuid "$facts"/sunshine-uuid
+    '';
+  };
+}

clanModules/syncthing/README.md

@@ -0,0 +1,32 @@
+A secure, file synchronization app for devices over networks, offering a private alternative to cloud services.
+---
+## Usage
+
+We recommend configuring this module as an sync-service through the provided options. Although it provides a Web GUI through which more usage scenarios are supported.
+
+## Features
+
+- **Private and Secure**: Syncthing uses TLS encryption to secure data transfer between devices, ensuring that only the intended devices can read your data.
+- **Decentralized**: No central server is involved in the data transfer. Each device communicates directly with others.
+- **Open Source**: The source code is openly available for audit and contribution, fostering trust and continuous improvement.
+- **Cross-Platform**: Syncthing supports multiple platforms including Windows, macOS, Linux, BSD, and Android.
+- **Real-time Synchronization**: Changes made to files are synchronized in real-time across all connected devices.
+- **Web GUI**: It includes a user-friendly web interface for managing devices and configurations. (`127.0.0.1:8384`)
+
+## Configuration
+
+- **Share Folders**: Select folders to share with connected devices and configure permissions and synchronization parameters.
+
+!!! info
+    Clan automatically discovers other devices. Automatic discovery requires one machine to be an [introducer](#clan.syncthing.introducer)
+
+    If that is not the case you can add the other device by its Device ID manually.
+    You can find and share Device IDs under the "Add Device" button in the Web GUI. (`127.0.0.1:8384`)
+
+## Troubleshooting
+
+- **Sync Conflicts**: Resolve synchronization conflicts manually by reviewing file versions and modification times in the Web GUI (`127.0.0.1:8384`).
+
+## Support
+
+- **Documentation**: Extensive documentation is available on the [Syncthing website](https://docs.syncthing.net/).

clanModules/syncthing/default.nix

@@ -1,14 +1,16 @@
-{ config
-, pkgs
-, lib
-, ...
+{
+  config,
+  pkgs,
+  lib,
+  ...
 }:
 {
   options.clan.syncthing = {
     id = lib.mkOption {
       type = lib.types.nullOr lib.types.str;
       example = "BABNJY4-G2ICDLF-QQEG7DD-N3OBNGF-BCCOFK6-MV3K7QJ-2WUZHXS-7DTW4AS";
-      default = config.clanCore.secrets.syncthing.facts."syncthing.pub".value or null;
+      default = config.clanCore.facts.services.syncthing.public."syncthing.pub".value or null;
+      defaultText = "config.clanCore.facts.services.syncthing.public.\"syncthing.pub\".value";
     };
     introducer = lib.mkOption {
       description = ''
@@ -32,6 +34,10 @@
       '';
       type = lib.types.listOf lib.types.str;
       default = [ ];
+      example = [
+        "folder1"
+        "folder2"
+      ];
     };
   };
 
@@ -53,17 +59,19 @@
 
       assertions = [
         {
-          assertion =
-            lib.all (attr: builtins.hasAttr attr config.services.syncthing.settings.folders)
-              config.clan.syncthing.autoShares;
+          assertion = lib.all (
+            attr: builtins.hasAttr attr config.services.syncthing.settings.folders
+          ) config.clan.syncthing.autoShares;
           message = ''
             Syncthing: If you want to AutoShare a folder, you need to have it configured on the sharing device.
           '';
         }
       ];
 
-      # Activates inofify compatibilty on syncthing
-      boot.kernel.sysctl."fs.inotify.max_user_watches" = lib.mkDefault 524288;
+      # Activates inotify compatibility on syncthing
+      # use mkOverride 900 here as it otherwise would collide with the default of the
+      # upstream nixos xserver.nix
+      boot.kernel.sysctl."fs.inotify.max_user_watches" = lib.mkOverride 900 524288;
 
       services.syncthing = {
         enable = true;
@@ -80,12 +88,8 @@
 
         group = "syncthing";
 
-        key =
-          lib.mkDefault
-            config.clan.secrets.syncthing.secrets."syncthing.key".path or null;
-        cert =
-          lib.mkDefault
-            config.clan.secrets.syncthing.secrets."syncthing.cert".path or null;
+        key = lib.mkDefault config.clan.secrets.syncthing.secrets."syncthing.key".path or null;
+        cert = lib.mkDefault config.clan.secrets.syncthing.secrets."syncthing.cert".path or null;
 
         settings = {
           options = {
@@ -115,7 +119,7 @@
           getPendingDevices = "/rest/cluster/pending/devices";
           postNewDevice = "/rest/config/devices";
           SharedFolderById = "/rest/config/folders/";
-          apiKey = config.clanCore.secrets.syncthing.secrets."syncthing.api".path or null;
+          apiKey = config.clanCore.facts.services.syncthing.secret."syncthing.api".path or null;
         in
         lib.mkIf config.clan.syncthing.autoAcceptDevices {
           description = "Syncthing auto accept devices";
@@ -127,51 +131,37 @@
             set -x
             # query pending deviceID's
             APIKEY=$(cat ${apiKey})
-            PENDING=$(${
-              lib.getExe pkgs.curl
-            } -X GET -H "X-API-Key: $APIKEY" ${baseAddress}${getPendingDevices})
+            PENDING=$(${lib.getExe pkgs.curl} -X GET -H "X-API-Key: $APIKEY" ${baseAddress}${getPendingDevices})
             PENDING=$(echo $PENDING | ${lib.getExe pkgs.jq} keys[])
 
             # accept pending deviceID's
             for ID in $PENDING;do
-            ${
-              lib.getExe pkgs.curl
-            } -X POST -d "{\"deviceId\": $ID}" -H "Content-Type: application/json" -H "X-API-Key: $APIKEY" ${baseAddress}${postNewDevice}
+            ${lib.getExe pkgs.curl} -X POST -d "{\"deviceId\": $ID}" -H "Content-Type: application/json" -H "X-API-Key: $APIKEY" ${baseAddress}${postNewDevice}
 
             # get all shared folders by their ID
             for folder in ${builtins.toString config.clan.syncthing.autoShares}; do
-              SHARED_IDS=$(${
-                lib.getExe pkgs.curl
-              } -X GET -H "X-API-Key: $APIKEY" ${baseAddress}${SharedFolderById}"$folder" | ${
-                lib.getExe pkgs.jq
-              } ."devices")
-              PATCHED_IDS=$(echo $SHARED_IDS | ${
-                lib.getExe pkgs.jq
-              } ".+= [{\"deviceID\": $ID, \"introducedBy\": \"\", \"encryptionPassword\": \"\"}]")
-              ${
-                lib.getExe pkgs.curl
-              } -X PATCH -d "{\"devices\": $PATCHED_IDS}" -H "X-API-Key: $APIKEY" ${baseAddress}${SharedFolderById}"$folder"
+              SHARED_IDS=$(${lib.getExe pkgs.curl} -X GET -H "X-API-Key: $APIKEY" ${baseAddress}${SharedFolderById}"$folder" | ${lib.getExe pkgs.jq} ."devices")
+              PATCHED_IDS=$(echo $SHARED_IDS | ${lib.getExe pkgs.jq} ".+= [{\"deviceID\": $ID, \"introducedBy\": \"\", \"encryptionPassword\": \"\"}]")
+              ${lib.getExe pkgs.curl} -X PATCH -d "{\"devices\": $PATCHED_IDS}" -H "X-API-Key: $APIKEY" ${baseAddress}${SharedFolderById}"$folder"
               done
             done
           '';
         };
 
-      systemd.timers.syncthing-auto-accept =
-        lib.mkIf config.clan.syncthing.autoAcceptDevices
-          {
-            description = "Syncthing Auto Accept";
+      systemd.timers.syncthing-auto-accept = lib.mkIf config.clan.syncthing.autoAcceptDevices {
+        description = "Syncthing Auto Accept";
 
-            wantedBy = [ "syncthing-auto-accept.service" ];
+        wantedBy = [ "syncthing-auto-accept.service" ];
 
-            timerConfig = {
-              OnActiveSec = lib.mkDefault 60;
-              OnUnitActiveSec = lib.mkDefault 60;
-            };
-          };
+        timerConfig = {
+          OnActiveSec = lib.mkDefault 60;
+          OnUnitActiveSec = lib.mkDefault 60;
+        };
+      };
 
       systemd.services.syncthing-init-api-key =
         let
-          apiKey = config.clanCore.secrets.syncthing.secrets."syncthing.api".path or null;
+          apiKey = config.clanCore.facts.services.syncthing.secret."syncthing.api".path or null;
         in
         lib.mkIf config.clan.syncthing.autoAcceptDevices {
           description = "Set the api key";
@@ -182,9 +172,7 @@
             set -efu pipefail
 
             APIKEY=$(cat ${apiKey})
-            ${
-              lib.getExe pkgs.gnused
-            } -i "s/<apikey>.*<\/apikey>/<apikey>$APIKEY<\/apikey>/" /var/lib/syncthing/config.xml
+            ${lib.getExe pkgs.gnused} -i "s/<apikey>.*<\/apikey>/<apikey>$APIKEY<\/apikey>/" /var/lib/syncthing/config.xml
             # sudo systemctl restart syncthing.service
             systemctl restart syncthing.service
           '';
@@ -195,11 +183,11 @@
           };
         };
 
-      clanCore.secrets.syncthing = {
-        secrets."syncthing.key" = { };
-        secrets."syncthing.cert" = { };
-        secrets."syncthing.api" = { };
-        facts."syncthing.pub" = { };
+      clanCore.facts.services.syncthing = {
+        secret."syncthing.key" = { };
+        secret."syncthing.cert" = { };
+        secret."syncthing.api" = { };
+        public."syncthing.pub" = { };
         generator.path = [
           pkgs.coreutils
           pkgs.gnugrep

clanModules/thelounge/README.md

@@ -0,0 +1,2 @@
+Modern web IRC client
+---

clanModules/thelounge/default.nix

@@ -0,0 +1,15 @@
+_: {
+  services.thelounge = {
+    enable = true;
+    public = true;
+    extraConfig = {
+      prefetch = true;
+      defaults = {
+        port = 6667;
+        tls = false;
+      };
+    };
+  };
+
+  clanCore.state.thelounde.folders = [ "/var/lib/thelounge" ];
+}

clanModules/trusted-nix-caches/README.md

@@ -0,0 +1,2 @@
+This module sets the `clan.lol` and `nix-community` cache up as a trusted cache.
+----

clanModules/trusted-nix-caches/default.nix

@@ -0,0 +1,10 @@
+{
+  nix.settings.trusted-substituters = [
+    "https://cache.clan.lol"
+    "https://nix-community.cachix.org"
+  ];
+  nix.settings.trusted-public-keys = [
+    "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
+    "cache.clan.lol-1:3KztgSAB5R1M+Dz7vzkBGzXdodizbgLXGXKXlcQLA28="
+  ];
+}

clanModules/user-password/README.md

@@ -0,0 +1,18 @@
+Automatically generates and configures a password for the specified user account.
+---
+
+If setting the option prompt to true, the user will be prompted to type in their desired password.
+
+!!! Note
+    This module will set `mutableUsers` to `false`, meaning you can not manage user passwords through `passwd` anymore.
+
+
+After the system was installed/deployed the following command can be used to display the user-password:
+
+```bash
+clan secrets get {machine_name}-user-password
+```
+
+See also: [Facts / Secrets](../../getting-started/secrets.md)
+
+To regenerate the password, delete the password files in the clan directory and redeploy the machine.

clanModules/user-password/default.nix

@@ -0,0 +1,49 @@
+{
+  pkgs,
+  config,
+  lib,
+  ...
+}:
+{
+  options.clan.user-password = {
+    user = lib.mkOption {
+      type = lib.types.str;
+      example = "alice";
+      description = "The user the password should be generated for.";
+    };
+    prompt = lib.mkOption {
+      type = lib.types.bool;
+      default = true;
+      example = false;
+      description = "Whether the user should be prompted.";
+    };
+  };
+
+  config = {
+    users.mutableUsers = false;
+    users.users.${config.clan.user-password.user}.hashedPasswordFile =
+      config.clanCore.facts.services.user-password.secret.user-password-hash.path;
+    sops.secrets."${config.clanCore.machineName}-user-password-hash".neededForUsers = true;
+    clanCore.facts.services.user-password = {
+      secret.user-password = { };
+      secret.user-password-hash = { };
+      generator.prompt = (
+        lib.mkIf config.clan.user-password.prompt "Set the password for your $user: ${config.clan.user-password.user}.
+      You can autogenerate a password, if you leave this prompt blank."
+      );
+      generator.path = with pkgs; [
+        coreutils
+        xkcdpass
+        mkpasswd
+      ];
+      generator.script = ''
+        if [[ -n $prompt_value ]]; then
+          echo $prompt_value > $secrets/user-password
+        else
+          xkcdpass --numwords 3 --delimiter - --count 1 > $secrets/user-password
+        fi
+        cat $secrets/user-password | mkpasswd -s -m sha-512 > $secrets/user-password-hash
+      '';
+    };
+  };
+}

clanModules/xfce/README.md

@@ -0,0 +1,2 @@
+A lightweight desktop manager
+---

clanModules/zerotier-static-peers/README.md

@@ -0,0 +1,5 @@
+Statically configure the `zerotier` peers of a clan network.
+---
+Statically configure the `zerotier` peers of a clan network.
+
+Requires a machine, that is the zerotier controller configured in the network.

clanModules/zerotier-static-peers/default.nix

@@ -0,0 +1,71 @@
+{
+  lib,
+  config,
+  pkgs,
+  inputs,
+  ...
+}:
+let
+  clanDir = config.clanCore.clanDir;
+  machineDir = clanDir + "/machines/";
+  machinesFileSet = builtins.readDir machineDir;
+  machines = lib.mapAttrsToList (name: _: name) machinesFileSet;
+
+  zerotierNetworkIdPath = machines: machineDir + machines + "/facts/zerotier-network-id";
+  networkIdsUnchecked = builtins.map (
+    machine:
+    let
+      fullPath = zerotierNetworkIdPath machine;
+    in
+    if builtins.pathExists fullPath then builtins.readFile fullPath else null
+  ) machines;
+  networkIds = lib.filter (machine: machine != null) networkIdsUnchecked;
+  networkId = builtins.elemAt networkIds 0;
+in
+#TODO:trace on multiple found network-ids
+#TODO:trace on no single found networkId
+{
+  options.clan.zerotier-static-peers = {
+    excludeHosts = lib.mkOption {
+      type = lib.types.listOf lib.types.str;
+      default = [ config.clanCore.machineName ];
+      description = "Hosts that should be excluded";
+    };
+  };
+
+  config.systemd.services.zerotier-static-peers-autoaccept =
+    let
+      machines = builtins.readDir machineDir;
+      zerotierIpMachinePath = machines: machineDir + machines + "/facts/zerotier-ip";
+      filteredMachines = lib.filterAttrs (
+        name: _: !(lib.elem name config.clan.static-hosts.excludeHosts)
+      ) machines;
+      hosts = lib.mapAttrsToList (host: _: host) (
+        lib.mapAttrs' (
+          machine: _:
+          let
+            fullPath = zerotierIpMachinePath machine;
+          in
+          if builtins.pathExists fullPath then
+            lib.nameValuePair (builtins.readFile fullPath) [ machine ]
+          else
+            null
+        ) filteredMachines
+      );
+    in
+    lib.mkIf (config.clan.networking.zerotier.controller.enable) {
+      wantedBy = [ "multi-user.target" ];
+      after = [ "zerotierone.service" ];
+      path = [ pkgs.zerotierone ];
+      serviceConfig.ExecStart = pkgs.writeScript "static-zerotier-peers-autoaccept" ''
+        #!/bin/sh
+        ${lib.concatMapStringsSep "\n" (host: ''
+          ${
+            inputs.clan-core.packages.${pkgs.system}.zerotier-members
+          }/bin/zerotier-members allow --member-ip ${host}
+        '') hosts}
+      '';
+    };
+
+  config.clan.networking.zerotier.networkId = lib.mkDefault networkId;
+}

clanModules/zt-tcp-relay/README.md

@@ -0,0 +1,2 @@
+Enable ZeroTier VPN over TCP for networks where UDP is blocked.
+---

clanModules/zt-tcp-relay/default.nix

@@ -1,4 +1,10 @@
-{ pkgs, lib, config, ... }: {
+{
+  pkgs,
+  lib,
+  config,
+  ...
+}:
+{
   options.clan.zt-tcp-relay = {
     port = lib.mkOption {
       type = lib.types.port;
@@ -13,7 +19,9 @@
       wantedBy = [ "multi-user.target" ];
       after = [ "network.target" ];
       serviceConfig = {
-        ExecStart = "${pkgs.callPackage ../pkgs/zt-tcp-relay {}}/bin/zt-tcp-relay --listen [::]:${builtins.toString config.clan.zt-tcp-relay.port}";
+        ExecStart = "${
+          pkgs.callPackage ../../pkgs/zt-tcp-relay { }
+        }/bin/zt-tcp-relay --listen [::]:${builtins.toString config.clan.zt-tcp-relay.port}";
         Restart = "always";
         RestartSec = "5";
         dynamicUsers = true;

devShell-python.nix

@@ -1,106 +0,0 @@
-{
-  perSystem =
-    { pkgs
-    , self'
-    , lib
-    , ...
-    }:
-    let
-      python3 = pkgs.python3;
-      pypkgs = python3.pkgs;
-      clan-cli = self'.packages.clan-cli;
-      clan-vm-manager = self'.packages.clan-vm-manager;
-      pythonWithDeps = python3.withPackages (
-        ps:
-        clan-cli.propagatedBuildInputs
-        ++ clan-cli.devDependencies
-        ++ [
-          ps.pip
-          # clan-vm-manager deps
-          ps.pygobject3
-        ]
-      );
-      linuxOnlyPackages = lib.optionals pkgs.stdenv.isLinux [
-        pkgs.xdg-utils
-      ];
-    in
-    {
-      devShells.python = pkgs.mkShell {
-        inputsFrom = [ self'.devShells.default ];
-        packages =
-          [
-            pythonWithDeps
-            pypkgs.mypy
-            pypkgs.ipdb
-            pkgs.desktop-file-utils
-            pkgs.gtk4.dev
-            pkgs.ruff
-            pkgs.libadwaita.devdoc # has the demo called 'adwaita-1-demo'
-          ]
-          ++ linuxOnlyPackages
-          ++ clan-vm-manager.nativeBuildInputs
-          ++ clan-vm-manager.buildInputs
-          ++ clan-cli.nativeBuildInputs;
-
-        PYTHONBREAKPOINT = "ipdb.set_trace";
-
-        shellHook = ''
-          ln -sfT ${clan-cli.nixpkgs} ./pkgs/clan-cli/clan_cli/nixpkgs
-
-          ## PYTHON
-
-          tmp_path=$(realpath ./.direnv)
-          repo_root=$(realpath .)
-          mkdir -p "$tmp_path/python/${pythonWithDeps.sitePackages}"
-
-          # local dependencies
-          localPackages=(
-            $repo_root/pkgs/clan-cli
-            $repo_root/pkgs/clan-vm-manager
-          )
-
-          # Install the package in editable mode
-          # This allows executing `clan` from within the dev-shell using the current
-          # version of the code and its dependencies.
-          # TODO: this is slow. get rid of pip or add better caching
-          echo "==== Installing local python packages in editable mode ===="
-          for package in "''${localPackages[@]}"; do
-            ${pythonWithDeps}/bin/pip install \
-              --quiet \
-              --disable-pip-version-check \
-              --no-index \
-              --no-build-isolation \
-              --prefix "$tmp_path/python" \
-              --editable "$package"
-          done
-
-          export PATH="$tmp_path/python/bin:$PATH"
-          export PYTHONPATH="''${PYTHONPATH:+$PYTHONPATH:}$tmp_path/python/${pythonWithDeps.sitePackages}"
-
-          for package in "''${localPackages[@]}"; do
-            export PYTHONPATH="$package:$PYTHONPATH"
-          done
-
-          if ! command -v xdg-mime &> /dev/null; then
-            echo "Warning: 'xdg-mime' is not available. The desktop file cannot be installed."
-          fi
-
-          # install desktop file
-          set -eou pipefail
-          DESKTOP_FILE_NAME=org.clan.vm-manager.desktop
-          DESKTOP_DST=~/.local/share/applications/$DESKTOP_FILE_NAME
-          DESKTOP_SRC=${clan-vm-manager.desktop-file}/share/applications/$DESKTOP_FILE_NAME
-          UI_BIN="clan-vm-manager"
-
-          cp -f $DESKTOP_SRC $DESKTOP_DST
-          sleep 2
-          sed -i "s|Exec=.*clan-vm-manager|Exec=$UI_BIN|" $DESKTOP_DST
-          xdg-mime default $DESKTOP_FILE_NAME  x-scheme-handler/clan
-          echo "==== Validating desktop file installation   ===="
-          set -x
-          desktop-file-validate $DESKTOP_DST
-          set +xeou pipefail
-        '';
-      };
-    };
-}

devShell.nix

@@ -1,9 +1,10 @@
 {
   perSystem =
-    { pkgs
-    , self'
-    , config
-    , ...
+    {
+      pkgs,
+      self',
+      config,
+      ...
     }:
     let
       writers = pkgs.callPackage ./pkgs/builders/script-writers.nix { };
@@ -16,16 +17,16 @@
       # A python program to switch between dev-shells
       # usage: select-shell shell-name
       # the currently enabled dev-shell gets stored in ./.direnv/selected-shell
-      select-shell = writers.writePython3Bin "select-shell"
-        {
-          flakeIgnore = [ "E501" ];
-        } ./pkgs/scripts/select-shell.py;
+      select-shell = writers.writePython3Bin "select-shell" {
+        flakeIgnore = [ "E501" ];
+      } ./pkgs/scripts/select-shell.py;
     in
     {
       devShells.default = pkgs.mkShell {
         packages = [
           select-shell
           pkgs.tea
+          pkgs.nix
           self'.packages.tea-create-pr
           self'.packages.merge-after-ci
           self'.packages.pending-reviews

docs/.envrc

@@ -0,0 +1,6 @@
+source_up
+
+watch_file $(find ./nix -name "*.nix" -printf '%p ')
+
+# Because we depend on nixpkgs sources, uploading to builders takes a long time
+use flake .#docs --builders ''

docs/.gitignore

@@ -0,0 +1 @@
+/site/reference

docs/CONTRIBUTING.md

@@ -153,3 +153,9 @@ If you need to inspect the Nix sandbox while running tests, follow these steps:
    cntr exec -w your_sandbox_name
    psgrep -a -x your_python_process_name
    ```
+
+
+# Standards
+
+Every new module name should be in kebab-case.
+Every fact definition, where possible should be in kebab-case.

docs/admins/_index.md

@@ -1,10 +0,0 @@
-+++
-title = "Admin Documentation"
-description = "Documentation administrators creating or managing cLANs"
-date = 2025-05-01T19:00:00+00:00
-updated = 2021-05-01T19:00:00+00:00
-template = "docs/section.html"
-weight = 15
-sort_by = "title"
-draft = false
-+++

docs/admins/clan-config.md

@@ -1,69 +0,0 @@
-# cLAN config
-
-`clan config` allows you to manage your nixos configuration via the terminal.
-Similar as how `git config` reads and sets git options, `clan config` does the same with your nixos options
-It also supports auto completion making it easy to find the right options.
-
-## Set up clan-config
-
-Add the clan tool to your flake inputs:
-
-```
-clan.url = "git+https://git.clan.lol/clan/clan-core";
-```
-
-and inside the mkFlake:
-
-```
-imports = [
-  inputs.clan.flakeModules.clan-config
-];
-```
-
-Add an empty config file and add it to git
-
-```command
-echo "{}" > ./clan-settings.json
-git add ./clan-settings.json
-```
-
-Import the clan-config module into your nixos configuration:
-
-```nix
-{
-  imports = [
-    # clan-settings.json is located in the same directory as your flake.
-    # Adapt the path if necessary.
-    (builtins.fromJSON (builtins.readFile ./clan-settings.json))
-  ];
-}
-
-
-```
-
-Make sure your nixos configuration is set a default
-
-```nix
-{self, ...}: {
-  flake.nixosConfigurations.default = self.nixosConfigurations.my-machine;
-}
-```
-
-Use all inputs provided by the clan-config devShell in your own devShell:
-
-```nix
-{ ... }: {
-  perSystem = { pkgs, self', ... }: {
-    devShells.default = pkgs.mkShell {
-      inputsFrom = [ self'.devShells.clan-config ];
-      # ...
-    };
-  };
-}
-```
-
-re-load your dev-shell to make the clan tool available.
-
-```command
-clan config --help
-```

docs/admins/machines.md

@@ -1,138 +0,0 @@
-# Managing NixOS Machines
-
-## Add Your First Machine
-
-To start managing a new machine, use the following commands to create and then list your machines:
-
-```shellSession
-$ clan machines create my-machine
-$ clan machines list
-my-machine
-```
-
-## Configure Your Machine
-
-In the example below, we demonstrate how to add a new user named `my-user` and set a password. This user will be configured to log in to the machine `my-machine`.
-
-### Creating a New User
-
-```shellSession
-# Add a new user
-$ clan config --machine my-machine users.users.my-user.isNormalUser true
-
-# Set a password for the user
-$ clan config --machine my-machine users.users.my-user.hashedPassword $(mkpasswd)
-```
-
-_Note: The `$(mkpasswd)` command generates a hashed password. Ensure you have the `mkpasswd` utility installed or use an alternative method to generate a secure hashed password._
-
-## Test Your Machine Configuration Inside a VM
-
-Before deploying your configuration to a live environment, you can run a virtual machine (VM) to test the settings:
-
-```shellSession
-$ clan vms run my-machine
-```
-
-This command run a VM based on the configuration of `my-machine`, allowing you to verify changes in a controlled environment.
-
-## Installing a New Machine
-
-Clan CLI, in conjunction with [nixos-anywhere](https://github.com/nix-community/nixos-anywhere), provides a seamless method for installing NixOS on various machines.
-This process involves preparing a suitable hardware and disk partitioning configuration and ensuring the target machine is accessible via SSH.
-
-### Prerequisites
-
-- A running Linux system with SSH on the target machine is required. This is typically pre-configured for many server providers.
-- For installations on physical hardware, create a NixOS installer image and transfer it to a bootable USB drive as described below.
-
-## Creating a Bootable USB Drive on Linux
-
-To create a bootable USB flash drive with the NixOS installer:
-
-1. **Build the Installer Image**:
-
-   ```shellSession
-   $ nix build git+https://git.clan.lol/clan/clan-core.git#install-iso
-   ```
-
-2. **Prepare the USB Flash Drive**:
-
-   - Insert your USB flash drive into your computer.
-   - Identify your flash drive with `lsblk`. Look for the device with a matching size.
-   - Ensure all partitions on the drive are unmounted. Replace `sdX` in the command below with your device identifier (like `sdb`, etc.):
-
-     ```shellSession
-     sudo umount /dev/sdX*
-     ```
-
-3. **Write the Image to the USB Drive**:
-
-   - Use the `dd` utility to write the NixOS installer image to your USB drive:
-
-     ```shellSession
-     sudo dd bs=4M conv=fsync oflag=direct status=progress if=./result/stick.raw of=/dev/sdX
-     ```
-
-4. **Boot and Connect**:
-   - After writing the installer to the USB drive, use it to boot the target machine.
-   - The installer will display an IP address and a root password, which you can use to connect via SSH.
-
-### Finishing the installation
-
-With the target machine running Linux and accessible via SSH, execute the following command to install NixOS on the target machine, replacing `<target_host>` with the machine's hostname or IP address:
-
-```shellSession
-$ clan machines install my-machine <target_host>
-```
-
-## Update Your Machines
-
-Clan CLI enables you to remotely update your machines over SSH. This requires setting up a target address for each target machine.
-
-### Setting the Target Host
-
-Replace `host_or_ip` with the actual hostname or IP address of your target machine:
-
-```shellSession
-$ clan config --machine my-machine clan.networking.targetHost root@host_or_ip
-```
-
-_Note: The use of `root@` in the target address implies SSH access as the root user.
-Ensure that the root login is secured and only used when necessary._
-
-### Updating Machine Configurations
-
-Execute the following command to update the specified machine:
-
-```shellSession
-$ clan machines update my-machine
-```
-
-You can also update all configured machines simultaneously by omitting the machine name:
-
-```shellSession
-$ clan machines update
-```
-
-### Setting a Build Host
-
-If the machine does not have enough resources to run the NixOS evaluation or build itself,
-it is also possible to specify a build host instead.
-During an update, the cli will ssh into the build host and run `nixos-rebuild` from there.
-
-```shellSession
-$ clan config --machine my-machine clan.networking.buildHost root@host_or_ip
-```
-
-### Excluding a machine from `clan machine update`
-
-To exclude machines from beeing updated when running `clan machines update` without any machines specified,
-one can set the `clan.deployment.requireExplicitUpdate` option to true:
-
-
-```shellSession
-$ clan config --machine my-machine clan.deployment.requireExplicitUpdate true
-```
-
-This is useful for machines that are not always online or are not part of the regular update cycle.

docs/admins/quickstart.md

@@ -1,135 +0,0 @@
-# Initializing a New Clan Project
-
-## Create a new flake
-
-1. To start a new project, execute the following command to add the clan cli to your shell:
-
-```shellSession
-$ nix shell git+https://git.clan.lol/clan/clan-core
-```
-
-2. Then use the following commands to initialize a new clan-flake:
-
-```shellSession
-$ clan flake create my-clan
-```
-
-This action will generate two primary files: `flake.nix` and `.clan-flake`.
-
-```shellSession
-$ ls -la
-drwx------ joerg users   5 B  a minute ago   ./
-drwxrwxrwt root  root  139 B  12 seconds ago ../
-.rw-r--r-- joerg users  77 B  a minute ago   .clan-flake
-.rw-r--r-- joerg users 4.8 KB a minute ago   flake.lock
-.rw-r--r-- joerg users 242 B  a minute ago   flake.nix
-```
-
-### Understanding the .clan-flake Marker File
-
-The `.clan-flake` marker file serves an optional purpose: it helps the `clan-cli` utility locate the project's root directory.
-If `.clan-flake` is missing, `clan-cli` will instead search for other indicators like `.git`, `.hg`, `.svn`, or `flake.nix` to identify the project root.
-
-## What's next
-
-After creating your flake, you can check out how to add [new machines](./machines.md)
-
----
-
-# Migrating Existing NixOS Configuration Flake
-
-Absolutely, let's break down the migration step by step, explaining each action in detail:
-
-#### Before You Begin
-
-1. **Backup Your Current Configuration**: Always start by making a backup of your current NixOS configuration to ensure you can revert if needed.
-
-   ```shellSession
-   $ cp -r /etc/nixos ~/nixos-backup
-   ```
-
-2. **Update Flake Inputs**: Add a new input for the `clan-core` dependency:
-
-   ```nix
-   inputs.clan-core = {
-     url = "git+https://git.clan.lol/clan/clan-core";
-     # Don't do this if your machines are on nixpkgs stable.
-     inputs.nixpkgs.follows = "nixpkgs";
-   };
-   ```
-
-   - `url`: Specifies the Git repository URL for Clan Core.
-   - `inputs.nixpkgs.follows`: Tells Nix to use the same `nixpkgs` input as your main input (in this case, it follows `nixpkgs`).
-
-3. **Update Outputs**: Then modify the `outputs` section of your `flake.nix` to adapt to Clan Core's new provisioning method. The key changes are as follows:
-
-   Add `clan-core` to the output
-
-   ```diff
-   -  outputs = { self, nixpkgs,  }:
-   +  outputs = { self, nixpkgs, clan-core }:
-   ```
-
-   Previous configuration:
-
-   ```nix
-   {
-       nixosConfigurations.example-desktop = nixpkgs.lib.nixosSystem {
-           system = "x86_64-linux";
-           modules = [
-               ./configuration.nix
-           ];
-           [...]
-       };
-   }
-   ```
-
-   After change:
-
-   ```nix
-   let clan = clan-core.lib.buildClan {
-       # this needs to point at the repository root
-       directory = self;
-       specialArgs = {};
-       clanName = "NEEDS_TO_BE_UNIQUE"; # TODO: Changeme
-       machines = {
-           example-desktop = {
-               nixpkgs.hostPlatform = "x86_64-linux";
-               imports = [
-                   ./configuration.nix
-               ];
-           };
-       };
-   };
-   in { inherit (clan) nixosConfigurations clanInternals; }
-   ```
-
-   - `nixosConfigurations`: Defines NixOS configurations, using Clan Core’s `buildClan` function to manage the machines.
-   - Inside `machines`, a new machine configuration is defined (in this case, `example-desktop`).
-   - Inside `example-desktop` which is the target machine hostname, `nixpkgs.hostPlatform` specifies the host platform as `x86_64-linux`.
-   - `clanInternals`: Is required to enable evaluation of the secret generation/upload script on every architecture
-   - `clanName`: Is required and needs to be globally unique, as else we have a cLAN name clash
-
-4. **Rebuild and Switch**: Rebuild your NixOS configuration using the updated flake:
-
-   ```shellSession
-   $ sudo nixos-rebuild switch --flake .
-   ```
-
-   - This command rebuilds and switches to the new configuration. Make sure to include the `--flake .` argument to use the current directory as the flake source.
-
-5. **Test Configuration**: Before rebooting, verify that your new configuration builds without errors or warnings.
-
-6. **Reboot**: If everything is fine, you can reboot your system to apply the changes:
-
-   ```shellSession
-   $ sudo reboot
-   ```
-
-7. **Verify**: After the reboot, confirm that your system is running with the new configuration, and all services and applications are functioning as expected.
-
-By following these steps, you've successfully migrated your NixOS Flake configuration to include the `clan-core` input and adapted the `outputs` section to work with Clan Core's new machine provisioning method.
-
-## What's next
-
-After creating your flake, you can check out how to add [new machines](./machines.md)

docs/admins/secrets-management.md

@@ -1,173 +0,0 @@
-# Managing Secrets with Clan
-
-Clan enables encryption of secrets within a Clan flake, ensuring secure sharing among users.
-This documentation will guide you through managing secrets with the Clan CLI,
-which utilizes the [sops](https://github.com/getsops/sops) format and
-integrates with [sops-nix](https://github.com/Mic92/sops-nix) on NixOS machines.
-
-## 1. Generating Keys and Creating Secrets
-
-To begin, generate a key pair:
-
-```shellSession
-$ clan secrets key generate
-```
-
-**Output**:
-
-```
-Public key: age1wkth7uhpkl555g40t8hjsysr20drq286netu8zptw50lmqz7j95sw2t3l7
-Generated age private key at '/home/joerg/.config/sops/age/keys.txt' for your user.
-Generated age private key at '/home/joerg/.config/sops/age/keys.txt' for your user. Please back it up on a secure location or you will lose access to your secrets.
-Also add your age public key to the repository with 'clan secrets users add youruser age1wkth7uhpkl555g40t8hjsysr20drq286netu8zptw50lmqz7j95sw2t3l7' (replace you
-user with your user name)
-```
-
-⚠️ **Important**: Backup the generated private key securely, or risk losing access to your secrets.
-
-Next, add your public key to the Clan flake repository:
-
-```shellSession
-$ clan secrets users add <your_username> <your_public_key>
-```
-
-Doing so creates this structure in your Clan flake:
-
-```
-sops/
-└── users/
-    └── <your_username>/
-        └── key.json
-```
-
-Now, to set your first secret:
-
-```shellSession
-$ clan secrets set mysecret
-Paste your secret:
-```
-
-Note: As you type your secret, keypresses won't be displayed. Press Enter to save the secret.
-
-Retrieve the stored secret:
-
-```shellSession
-$ clan secrets get mysecret
-```
-
-And list all secrets like this:
-
-```shellSession
-$ clan secrets list
-```
-
-Secrets in the repository follow this structure:
-
-```
-sops/
-├── secrets/
-│   └── <secret_name>/
-│       ├── secret
-│       └── users/
-│           └── <your_username>/
-```
-
-The content of the secret is stored encrypted inside the `secret` file under `mysecret`.
-By default, secrets are encrypted with your key to ensure readability.
-
-## 2. Adding Machine Keys
-
-New machines in Clan come with age keys stored in `./sops/machines/<machine_name>`. To list these machines:
-
-```shellSession
-$ clan secrets machines list
-```
-
-For existing machines, add their keys:
-
-```shellSession
-$ clan secrets machines add <machine_name> <age_key>
-```
-
-To fetch an age key from an SSH host key:
-
-```shellSession
-$ ssh-keyscan <domain_name> | nix shell nixpkgs#ssh-to-age -c ssh-to-age
-```
-
-## 3. Assigning Access
-
-By default, secrets are encrypted for your key. To specify which users and machines can access a secret:
-
-```shellSession
-$ clan secrets set --machine <machine1> --machine <machine2> --user <user1> --user <user2> <secret_name>
-```
-
-You can add machines/users to existing secrets without modifying the secret:
-
-```shellSession
-$ clan secrets machines add-secret <machine_name> <secret_name>
-```
-
-## 4. Utilizing Groups
-
-For convenience, Clan CLI allows group creation to simplify access management. Here's how:
-
-1. **Creating Groups**:
-
-   Assign users to a new group, e.g., `admins`:
-
-   ```shellSession
-   $ clan secrets groups add admins <username>
-   ```
-
-2. **Listing Groups**:
-
-   ```shellSession
-   $ clan secrets groups list
-   ```
-
-3. **Assigning Secrets to Groups**:
-
-   ```shellSession
-   $ clan secrets groups add-secret <group_name> <secret_name>
-   ```
-
-# NixOS integration
-
-A NixOS machine will automatically import all secrets that are encrypted for the
-current machine. At runtime it will use the host key to decrypt all secrets into
-a in-memory, non-persistent filesystem using
-[sops-nix](https://github.com/Mic92/sops-nix). In your nixos configuration you
-can get a path to secrets like this `config.sops.secrets.<name>.path`. Example:
-
-```nix
-{ config, ...}: {
-  sops.secrets.my-password.neededForUsers = true;
-
-  users.users.mic92 = {
-    isNormalUser = true;
-    passwordFile = config.sops.secrets.my-password.path;
-  };
-}
-```
-
-See the [readme](https://github.com/Mic92/sops-nix) of sops-nix for more
-examples.
-
-# Importing existing sops-based keys / sops-nix
-
-`clan secrets` stores each secrets in a single file, whereas [sops](https://github.com/Mic92/sops-nix)
-commonly allows to put all secrets in a yaml or json documents.
-
-If you already happend to use sops-nix, you can migrate by using the `clan secrets import-sops` command by importing these documents:
-
-```shellSession
-% clan secrets import-sops --prefix matchbox- --group admins --machine matchbox nixos/matchbox/secrets/secrets.yaml
-```
-
-This will create secrets for each secret found in `nixos/matchbox/secrets/secrets.yaml` in a ./sops folder of your repository.
-Each member of the group `admins` will be able
-
-Since our clan secret module will auto-import secrets that are encrypted for a particular nixos machine,
-you can now remove `sops.secrets.<secrets> = { };` unless you need to specify more options for the secret like owner/group of the secret file.

docs/admins/zerotier.md

@@ -1,69 +0,0 @@
-# ZeroTier Configuration with NixOS in Clan
-
-This guide provides detailed instructions for configuring
-[ZeroTier VPN](https://zerotier.com) within Clan. Follow the
-outlined steps to set up a machine as a VPN controller (`<CONTROLLER>`) and to
-include a new machine into the VPN.
-
-## 1. Setting Up the VPN Controller
-
-The VPN controller is initially essential for providing configuration to new
-peers. Post the address allocation, the controller's continuous operation is not
-crucial.
-
-### Instructions:
-
-1. **Designate a Machine**: Label a machine as the VPN controller in the clan,
-   referred to as `<CONTROLLER>` henceforth in this guide.
-2. **Add Configuration**: Input the below configuration to the NixOS
-   configuration of the controller machine:
-   ```nix
-   clan.networking.zerotier.controller = {
-     enable = true;
-     public = true;
-   };
-   ```
-3. **Update the Controller Machine**: Execute the following:
-   ```console
-   $ clan machines update <CONTROLLER>
-   ```
-   Your machine is now operational as the VPN controller.
-
-## 2. Integrating a New Machine to the VPN
-
-To introduce a new machine to the VPN, adhere to the following steps:
-
-### Instructions:
-
-1. **Update Configuration**: On the new machine, incorporate the below to its
-   configuration, substituting `<CONTROLLER>` with the controller machine name:
-   ```nix
-   { config, ... }: {
-     clan.networking.zerotier.networkId = builtins.readFile (config.clanCore.clanDir + "/machines/<CONTROLLER>/facts/zerotier-network-id");
-   }
-   ```
-2. **Update the New Machine**: Execute:
-   ```console
-   $ clan machines update <NEW_MACHINE>
-   ```
-   Replace `<NEW_MACHINE>` with the designated new machine name.
-3. **Retrieve the ZeroTier ID**: On the `new_machine`, execute:
-   ```console
-   $ sudo zerotier-cli info
-   ```
-   Example Output: `200 info d2c71971db 1.12.1 OFFLINE`, where `d2c71971db` is
-   the ZeroTier ID.
-4. **Authorize the New Machine on Controller**: On the controller machine,
-   execute:
-   ```console
-   $ sudo zerotier-members allow <ID>
-   ```
-   Substitute `<ID>` with the ZeroTier ID obtained previously.
-5. **Verify Connection**: On the `new_machine`, re-execute:
-   ```console
-   $ sudo zerotier-cli info
-   ```
-   The status should now be "ONLINE" e.g., `200 info 47303517ef 1.12.1 ONLINE`.
-
-Congratulations! The new machine is now part of the VPN, and the ZeroTier
-configuration on NixOS within the Clan project is complete.

docs/main.py

@@ -0,0 +1,30 @@
+from typing import Any
+
+
+def define_env(env: Any) -> None:
+    static_dir = "/static/"
+    video_dir = "https://clan.lol/" + "videos/"
+    asciinema_dir = static_dir + "asciinema-player/"
+
+    @env.macro
+    def video(name: str) -> str:
+        return f"""<video loop muted autoplay id="{name}">
+                <source src={video_dir + name} type="video/webm">
+                Your browser does not support the video tag.
+            </video>"""
+
+    @env.macro
+    def asciinema(name: str) -> str:
+        return f"""<div id="{name}">
+            <script src="{asciinema_dir}/asciinema-player.min.js"></script>
+            <script>
+                AsciinemaPlayer.create('{video_dir + name}',
+                document.getElementById("{name}"), {{
+                    loop: true,
+                    autoPlay: true,
+                    controls: false,
+                    speed: 1.5,
+                    theme: "solarized-light"
+                }});
+            </script>
+        </div>"""

docs/mkdocs.yml

@@ -0,0 +1,160 @@
+site_name: Clan Docs
+site_url: https://docs.clan.lol
+repo_url: https://git.clan.lol/clan/clan-core/
+repo_name: clan-core
+edit_uri: _edit/main/docs/docs/
+
+validation:
+  omitted_files: warn
+  absolute_links: warn
+  unrecognized_links: warn
+
+markdown_extensions:
+  - admonition
+  - attr_list
+  - footnotes
+  - md_in_html
+  - meta
+  - plantuml_markdown
+  - pymdownx.emoji:
+      emoji_index: !!python/name:material.extensions.emoji.twemoji
+      emoji_generator: !!python/name:material.extensions.emoji.to_svg
+  - pymdownx.tasklist:
+      custom_checkbox: true
+  - pymdownx.superfences
+  - pymdownx.tabbed:
+      alternate_style: true
+  - pymdownx.details
+  - pymdownx.highlight:
+      use_pygments: true
+      anchor_linenums: true
+  - toc:
+      title: On this page
+
+exclude_docs: |
+  .*
+  !templates/
+  /drafts/
+
+nav:
+  - Blog:
+    - blog/index.md
+  - Getting started:
+      - index.md
+      - Installer: getting-started/installer.md
+      - Configure: getting-started/configure.md
+      - Secrets & Facts: getting-started/secrets.md
+      - Deploy Machine: getting-started/deploy.md
+      - Mesh VPN: getting-started/mesh-vpn.md
+      - Backup & Restore: getting-started/backups.md
+      - Flake-parts: getting-started/flake-parts.md
+  - Modules:
+      - Clan Modules:
+          - reference/clanModules/borgbackup.md
+          - reference/clanModules/deltachat.md
+          - reference/clanModules/disk-layouts.md
+          - reference/clanModules/ergochat.md
+          - reference/clanModules/localbackup.md
+          - reference/clanModules/localsend.md
+          - reference/clanModules/matrix-synapse.md
+          - reference/clanModules/moonlight.md
+          - reference/clanModules/root-password.md
+          - reference/clanModules/sshd.md
+          - reference/clanModules/sunshine.md
+          - reference/clanModules/syncthing.md
+          - reference/clanModules/static-hosts.md
+          - reference/clanModules/thelounge.md
+          - reference/clanModules/trusted-nix-caches.md
+          - reference/clanModules/user-password.md
+          - reference/clanModules/xfce.md
+          - reference/clanModules/zerotier-static-peers.md
+          - reference/clanModules/zt-tcp-relay.md
+      - CLI:
+          - reference/cli/index.md
+          - reference/cli/backups.md
+          - reference/cli/config.md
+          - reference/cli/facts.md
+          - reference/cli/flakes.md
+          - reference/cli/flash.md
+          - reference/cli/history.md
+          - reference/cli/machines.md
+          - reference/cli/secrets.md
+          - reference/cli/ssh.md
+          - reference/cli/vms.md
+      - Clan Core:
+          - reference/clan-core/index.md
+          - reference/clan-core/backups.md
+          - reference/clan-core/facts.md
+          - reference/clan-core/sops.md
+          - reference/clan-core/state.md
+  - Contributing: contributing/contributing.md
+
+docs_dir: site
+site_dir: out
+
+theme:
+  logo: static/clan-white.png
+  favicon: static/clan-dark.png
+  name: material
+  features:
+    - navigation.instant
+    - navigation.tabs
+    - content.code.annotate
+    - content.code.copy
+    - content.tabs.link
+  icon:
+    repo: fontawesome/brands/git
+  font:
+    code: Roboto Mono
+
+  palette:
+    # Palette toggle for light mode
+    - media: "(prefers-color-scheme: light)"
+      scheme: default
+      primary: teal
+      accent: deep purple
+      toggle:
+        icon: material/weather-night
+        name: Switch to dark mode
+
+    # Palette toggle for dark mode
+    - media: "(prefers-color-scheme: dark)"
+      primary: teal
+      accent: deep purple
+      scheme: slate
+      toggle:
+        icon: material/weather-sunny
+        name: Switch to light mode
+
+extra_css:
+  - static/asciinema-player/custom-theme.css
+  - static/asciinema-player/asciinema-player.css
+
+extra:
+  social:
+    - icon: fontawesome/regular/comment
+      link: https://matrix.to/#/#clan:lassul.us
+    - icon: fontawesome/brands/gitlab
+      link: https://git.clan.lol/clan/clan-core
+    - icon: fontawesome/brands/github
+      link: https://github.com/clan-lol/clan-core
+    - icon: fontawesome/solid/rss
+      link: /feed_rss_created.xml
+
+
+plugins:
+  - search
+  - blog
+  - macros
+  - rss:
+      match_path: blog/posts/.*
+      use_git: false
+      date_from_meta:
+        as_creation: "date"
+        as_update: "date"
+        datetime_format: "%Y-%m-%d %H:%M"
+        default_timezone: Europe/Paris
+        default_time: "17:18"
+      categories:
+        - categories
+        - tags

docs/nix/default.nix

@@ -0,0 +1,40 @@
+{
+  pkgs,
+  module-docs,
+  clan-cli-docs,
+  ...
+}:
+let
+  uml-c4 = pkgs.python3Packages.plantuml-markdown.override { plantuml = pkgs.plantuml-c4; };
+in
+pkgs.stdenv.mkDerivation {
+  name = "clan-documentation";
+
+  src = ../.;
+
+  nativeBuildInputs =
+    [
+      pkgs.python3
+      uml-c4
+    ]
+    ++ (with pkgs.python3Packages; [
+      mkdocs
+      mkdocs-material
+      mkdocs-rss-plugin
+      mkdocs-macros
+    ]);
+  configurePhase = ''
+    mkdir -p ./site/reference/cli
+    cp -af ${module-docs}/* ./site/reference/
+    cp -af ${clan-cli-docs}/* ./site/reference/cli/
+  '';
+
+  buildPhase = ''
+    mkdocs build --strict
+    ls -la .
+  '';
+
+  installPhase = ''
+    cp -a out/ $out/
+  '';
+}

docs/nix/deploy-docs.nix

@@ -0,0 +1,49 @@
+{
+  writeShellScriptBin,
+  coreutils,
+  openssh,
+  rsync,
+  lib,
+  docs,
+}:
+
+writeShellScriptBin "deploy-docs" ''
+  set -eu -o pipefail
+  export PATH="${
+    lib.makeBinPath [
+      coreutils
+      openssh
+      rsync
+    ]
+  }"
+
+  #########################################
+  #                                       #
+  # DO NOT PRINT THE SSH KEY TO THE LOGS  #
+  #                                       #
+  #########################################
+  set +x
+  if [ -n "''${SSH_HOMEPAGE_KEY:-}" ]; then
+    echo "$SSH_HOMEPAGE_KEY" > ./ssh_key
+    chmod 600 ./ssh_key
+    sshExtraArgs="-i ./ssh_key"
+  else
+    sshExtraArgs=
+  fi
+  set -x
+  ###########################
+  #                         #
+  #    END OF DANGER ZONE   #
+  #                         #
+  ###########################
+
+
+  rsync \
+    -e "ssh -o StrictHostKeyChecking=no $sshExtraArgs" \
+    -a ${docs}/ \
+    www@clan.lol:/var/www/docs.clan.lol
+
+  if [ -e ./ssh_key ]; then
+    rm ./ssh_key
+  fi
+''

docs/nix/flake-module.nix

@@ -0,0 +1,70 @@
+{ inputs, self, ... }:
+{
+  perSystem =
+    {
+      config,
+      self',
+      pkgs,
+      ...
+    }:
+    let
+      # Simply evaluated options (JSON)
+      # { clanCore = «derivation JSON»; clanModules = { ${name} = «derivation JSON» }; }
+      jsonDocs = import ./get-module-docs.nix {
+        inherit (inputs) nixpkgs;
+        inherit pkgs self;
+        inherit (self.nixosModules) clanCore;
+        inherit (self) clanModules;
+      };
+
+      clanModulesFileInfo = pkgs.writeText "info.json" (builtins.toJSON jsonDocs.clanModules);
+      clanModulesReadmes = pkgs.writeText "info.json" (builtins.toJSON jsonDocs.clanModulesReadmes);
+
+      # Simply evaluated options (JSON)
+      renderOptions =
+        pkgs.runCommand "renderOptions.py"
+          {
+            # TODO: ruff does not splice properly in nativeBuildInputs
+            depsBuildBuild = [ pkgs.ruff ];
+            nativeBuildInputs = [
+              pkgs.python3
+              pkgs.mypy
+            ];
+          }
+          ''
+            install ${./scripts/renderOptions.py} $out
+            patchShebangs --build $out
+
+            ruff format --check --diff $out
+            ruff --line-length 88 $out
+            mypy --strict $out
+          '';
+
+      module-docs = pkgs.runCommand "rendered" { nativeBuildInputs = [ pkgs.python3 ]; } ''
+        export CLAN_CORE=${jsonDocs.clanCore}/share/doc/nixos/options.json 
+        # A file that contains the links to all clanModule docs
+        export CLAN_MODULES=${clanModulesFileInfo}
+        export CLAN_MODULES_READMES=${clanModulesReadmes}
+
+        mkdir $out
+
+        # The python script will place mkDocs files in the output directory
+        python3 ${renderOptions}
+      '';
+    in
+    {
+      devShells.docs = pkgs.callPackage ./shell.nix {
+        inherit (self'.packages) docs clan-cli-docs;
+        inherit module-docs;
+      };
+      packages = {
+        docs = pkgs.python3.pkgs.callPackage ./default.nix {
+          inherit (self'.packages) clan-cli-docs;
+          inherit (inputs) nixpkgs;
+          inherit module-docs;
+        };
+        deploy-docs = pkgs.callPackage ./deploy-docs.nix { inherit (config.packages) docs; };
+        inherit module-docs;
+      };
+    };
+}

docs/nix/get-module-docs.nix

@@ -0,0 +1,51 @@
+{
+  nixpkgs,
+  pkgs,
+  clanCore,
+  clanModules,
+  self,
+}:
+let
+  allNixosModules = (import "${nixpkgs}/nixos/modules/module-list.nix") ++ [
+    "${nixpkgs}/nixos/modules/misc/assertions.nix"
+    { nixpkgs.hostPlatform = "x86_64-linux"; }
+  ];
+
+  clanCoreNixosModules = [
+    clanCore
+    { clanCore.clanDir = ./.; }
+  ] ++ allNixosModules;
+
+  # TODO: optimally we would not have to evaluate all nixos modules for every page
+  #   but some of our module options secretly depend on nixos modules.
+  #   We would have to get rid of these implicit dependencies and make them explicit
+  clanCoreNixos = pkgs.nixos { imports = clanCoreNixosModules; };
+
+  # using extendModules here instead of re-evaluating nixos every time
+  #   improves eval performance slightly (10%)
+  getOptions = modules: (clanCoreNixos.extendModules { inherit modules; }).options;
+
+  evalDocs =
+    options:
+    pkgs.nixosOptionsDoc {
+      options = options;
+      warningsAreErrors = false;
+    };
+
+  # clanModules docs
+  clanModulesDocs = builtins.mapAttrs (
+    name: module: (evalDocs ((getOptions [ module ]).clan.${name} or { })).optionsJSON
+  ) clanModules;
+
+  clanModulesReadmes = builtins.mapAttrs (
+    module_name: _module: self.lib.modules.getReadme module_name
+  ) clanModules;
+
+  # clanCore docs
+  clanCoreDocs = (evalDocs (getOptions [ ]).clanCore).optionsJSON;
+in
+{
+  inherit clanModulesReadmes;
+  clanCore = clanCoreDocs;
+  clanModules = clanModulesDocs;
+}

docs/nix/scripts/renderOptions.py

@@ -0,0 +1,235 @@
+# Options are available in the following format:
+# https://github.com/nixos/nixpkgs/blob/master/nixos/lib/make-options-doc/default.nix
+#
+# ```json
+# {
+# ...
+# "fileSystems.<name>.options": {
+#     "declarations": ["nixos/modules/tasks/filesystems.nix"],
+#     "default": {
+#       "_type": "literalExpression",
+#       "text": "[\n  \"defaults\"\n]"
+#     },
+#     "description": "Options used to mount the file system.",
+#     "example": {
+#       "_type": "literalExpression",
+#       "text": "[\n  \"data=journal\"\n]"
+#     },
+#     "loc": ["fileSystems", "<name>", "options"],
+#     "readOnly": false,
+#     "type": "non-empty (list of string (with check: non-empty))"
+#     "relatedPackages": "- [`pkgs.tmux`](\n    https://search.nixos.org/packages?show=tmux&sort=relevance&query=tmux\n  )\n",
+# }
+# }
+# ```
+
+import json
+import os
+from pathlib import Path
+from typing import Any
+
+# Get environment variables
+CLAN_CORE = os.getenv("CLAN_CORE")
+CLAN_MODULES = os.environ.get("CLAN_MODULES")
+CLAN_MODULES_READMES = os.environ.get("CLAN_MODULES_READMES")
+
+OUT = os.environ.get("out")
+
+
+def sanitize(text: str) -> str:
+    return text.replace(">", "\\>")
+
+
+def replace_store_path(text: str) -> Path:
+    res = text
+    if text.startswith("/nix/store/"):
+        res = "https://git.clan.lol/clan/clan-core/src/branch/main/" + str(
+            Path(*Path(text).parts[4:])
+        )
+    return Path(res)
+
+
+def render_option_header(name: str) -> str:
+    return f"# {name}\n"
+
+
+def join_lines_with_indentation(lines: list[str], indent: int = 4) -> str:
+    """
+    Joins multiple lines with a specified number of whitespace characters as indentation.
+
+    Args:
+    lines (list of str): The lines of text to join.
+    indent (int): The number of whitespace characters to use as indentation for each line.
+
+    Returns:
+    str: The indented and concatenated string.
+    """
+    # Create the indentation string (e.g., four spaces)
+    indent_str = " " * indent
+    # Join each line with the indentation added at the beginning
+    return "\n".join(indent_str + line for line in lines)
+
+
+def render_option(name: str, option: dict[str, Any], level: int = 3) -> str:
+    read_only = option.get("readOnly")
+
+    res = f"""
+{"#" * level} {sanitize(name)}
+{"Readonly" if read_only else ""}
+{option.get("description", "No description available.")}
+
+**Type**: `{option["type"]}`
+
+"""
+    if option.get("default"):
+        res += f"""
+**Default**:
+
+```nix
+{option["default"]["text"] if option.get("default") else "No default set."}
+```
+        """
+    example = option.get("example", {}).get("text")
+    if example:
+        example_indented = join_lines_with_indentation(example.split("\n"))
+        res += f"""
+
+???+ example
+
+    ```nix
+{example_indented}
+    ```
+"""
+    if option.get("relatedPackages"):
+        res += f"""
+### Related Packages
+
+{option["relatedPackages"]}
+"""
+
+    decls = option.get("declarations", [])
+    source_path = replace_store_path(decls[0])
+    res += f"""
+:simple-git: [{source_path.name}]({source_path})
+"""
+    res += "\n"
+
+    return res
+
+
+def module_header(module_name: str) -> str:
+    return f"# {module_name}\n"
+
+
+def module_usage(module_name: str) -> str:
+    return f"""## Usage
+
+To use this module, import it like this:
+
+```nix
+{{config, lib, inputs, ...}}: {{
+    imports = [ inputs.clan-core.clanModules.{module_name} ];
+    # ...
+}}
+```
+"""
+
+
+clan_core_descr = """ClanCore delivers all the essential features for every clan. 
+It's always included in your setup, and you can customize your clan's behavior with the configuration [options](#module-options) provided below.
+
+"""
+
+options_head = "\n## Module Options\n"
+
+
+def produce_clan_core_docs() -> None:
+    if not CLAN_CORE:
+        raise ValueError(
+            f"Environment variables are not set correctly: $CLAN_CORE={CLAN_CORE}"
+        )
+
+    if not OUT:
+        raise ValueError(f"Environment variables are not set correctly: $out={OUT}")
+
+    # A mapping of output file to content
+    core_outputs: dict[str, str] = {}
+    with open(CLAN_CORE) as f:
+        options: dict[str, dict[str, Any]] = json.load(f)
+        module_name = "clan-core"
+        for option_name, info in options.items():
+            outfile = f"{module_name}/index.md"
+
+            # Create seperate files for nested options
+            if len(option_name.split(".")) <= 2:
+                # i.e. clan-core.clanDir
+                output = core_outputs.get(
+                    outfile,
+                    module_header(module_name) + clan_core_descr + options_head,
+                )
+                output += render_option(option_name, info)
+                # Update the content
+                core_outputs[outfile] = output
+            else:
+                # Clan sub-options
+                [_, sub] = option_name.split(".")[0:2]
+                outfile = f"{module_name}/{sub}.md"
+                # Get the content or write the header
+                output = core_outputs.get(outfile, render_option_header(sub))
+                output += render_option(option_name, info)
+                # Update the content
+                core_outputs[outfile] = output
+
+        for outfile, output in core_outputs.items():
+            (Path(OUT) / outfile).parent.mkdir(parents=True, exist_ok=True)
+            with open(Path(OUT) / outfile, "w") as of:
+                of.write(output)
+
+
+def produce_clan_modules_docs() -> None:
+    if not CLAN_MODULES:
+        raise ValueError(
+            f"Environment variables are not set correctly: $CLAN_MODULES={CLAN_MODULES}"
+        )
+    if not CLAN_MODULES_READMES:
+        raise ValueError(
+            f"Environment variables are not set correctly: $CLAN_MODULES_READMES={CLAN_MODULES_READMES}"
+        )
+
+    if not OUT:
+        raise ValueError(f"Environment variables are not set correctly: $out={OUT}")
+
+    with open(CLAN_MODULES) as f:
+        links: dict[str, str] = json.load(f)
+
+    with open(CLAN_MODULES_READMES) as readme:
+        readme_map: dict[str, str] = json.load(readme)
+
+    # {'borgbackup': '/nix/store/hi17dwgy7963ddd4ijh81fv0c9sbh8sw-options.json', ... }
+    for module_name, options_file in links.items():
+        with open(Path(options_file) / "share/doc/nixos/options.json") as f:
+            options: dict[str, dict[str, Any]] = json.load(f)
+            print(f"Rendering options for {module_name}...")
+            output = module_header(module_name)
+
+            if readme_map.get(module_name, None):
+                output += f"{readme_map[module_name]}\n"
+
+            output += module_usage(module_name)
+
+            output += options_head if len(options.items()) else ""
+            for option_name, info in options.items():
+                output += render_option(option_name, info)
+
+            outfile = Path(OUT) / f"clanModules/{module_name}.md"
+            outfile.parent.mkdir(
+                parents=True,
+                exist_ok=True,
+            )
+            with open(outfile, "w") as of:
+                of.write(output)
+
+
+if __name__ == "__main__":
+    produce_clan_core_docs()
+    produce_clan_modules_docs()

docs/nix/shell.nix

@@ -0,0 +1,18 @@
+{
+  docs,
+  pkgs,
+  module-docs,
+  clan-cli-docs,
+  ...
+}:
+pkgs.mkShell {
+  inputsFrom = [ docs ];
+  shellHook = ''
+    mkdir -p ./site/reference/cli
+    cp -af ${module-docs}/* ./site/reference/
+    cp -af ${clan-cli-docs}/* ./site/reference/cli/
+    chmod +w ./site/reference/*
+
+    echo "Generated API documentation in './site/reference/' "
+  '';
+}

docs/site/blog/.authors.yml

@@ -0,0 +1,21 @@
+authors:
+  Lassulus:
+    name: "Lassulus"
+    description: "Contributor to Clan"
+    avatar: "https://avatars.githubusercontent.com/u/621759?v=4"
+    url: "https://http://lassul.us/"
+  Mic92:
+    name: "Mic92"
+    description: "Contributor to Clan"
+    avatar: "https://avatars.githubusercontent.com/u/96200?v=4"
+    url: "https://thalheim.io"
+  W:
+    name: "W"
+    description: "Founder of Clan"
+    avatar: "/static/w_profile.webp"
+    url: ""
+  Qubasa:
+    name: "Qubasa"
+    description: "Contributor to Clan"
+    avatar: "https://avatars.githubusercontent.com/u/22085373?v=4"
+    url: "https://github.com/Qubasa"

docs/site/blog/index.md

@@ -0,0 +1,2 @@
+# Blog
+

docs/site/blog/posts/hello-world.md

@@ -0,0 +1,72 @@
+---
+title: "Introducing Clan: Full-Stack Computing Redefined"
+description: "Introducing Clan, a new model for a decentralized network, designed to provide families, smaller groups, and small businesses a platform that’s private, secure, and user-friendly."
+authors:
+  - W
+  - Qubasa
+date: 2024-03-19
+---
+
+
+In a digital age where users are guided increasingly toward submission and dependence, Clan reclaims computing and networking from the ground up.
+
+Clan enables users to build any system from a git repository, automate secret handling, and join devices in a secure darknet. This control extends beyond applications to communication protocols and the operating system itself, putting you fully in charge of your own digital environment.
+
+## Why We're Building Clan
+
+Our mission is simple: to restore fun, freedom, and functionality to computing as an open source project. We believe in building tools that empower users, foster innovation, and challenge the limitations imposed by outdated paradigms. Clan, in its essence, is an open source endeavor; it's our contribution to a future where technology serves humanity, not the other way around.
+
+## How Clan Changes the Game
+Clan embodies a new philosophy in system, application, and network design. It enables seamless, secure communication across devices, simplifies software distribution and updates, and offers both public and private network configurations. Here are some of the ways it accomplishes this:
+
+- **Nix as a Foundation:** Imagine a safety net for your computer's operating system, one that lets you make changes or updates without the fear of causing a crash or losing data. Nix simplifies the complexities of system design, ensuring that updates are safe and systems are more reliable.
+
+- **Simplified System Deployment:** Building and managing a computer system, from the operating system to the software you use, often feels like putting together a complex puzzle. With Clan, the puzzle pieces are replaced by a set of building blocks. Leveraging the power of Nix and Clan's innovative toolkit, anyone from tech-savvy administrators to everyday users can create and maintain what we call "full-stack systems" (everything your computer needs to run smoothly).
+
+- **A Leap in Connectivity:** Imagine if you could create private, secure pathways between your devices, bypassing the noisy and often insecure internet. Clan makes this possible through something called "overlay networks." These networks are like private tunnels, allowing your devices to talk to each other securely and directly. With Clan's built-in overlay networks and automatically configured services, connecting your devices becomes seamless, secure, and hassle-free.
+
+- **Security Through Separation:** Clan employs sandboxing and virtual machines, a technology that runs code in isolated environments - so even if you explore new Clans, your system remains protected from potential threats.
+
+- **Reliable:** With Clan, your data and services are preserved for the long haul. We focus on self-hosted backups and integration with the [Fediverse](https://de.wikipedia.org/wiki/Fediverse), a network of interconnected, independent online communities, so your digital life remains uninterrupted and under your control.
+
+
+## A Glimpse at Clan's Features
+
+- **Social Scaling:** Choose between creating a private sanctuary for your closest contacts, a dynamic space for a self-contained community, or embracing the open web with public Clans anyone can join.
+
+{{ video(name="show_join.webm")}}
+
+- **Seamless VM Integration:** Applications running in virtual machines can appear and behave as if they're part of your main operating system — a blend of power and simplicity.
+
+{{ video(name="show_run.webm")}}
+
+- **Robust Backup Management:** Keep your data safe _forever_ - never worry about cloud services disappearing in 10 years.
+
+{{ asciinema(name="backups.cast") }}
+
+- **Intuitive Secret Management:** Clan simplifies digital security by automating the creation and management of encryption keys and passwords for your services.
+
+{{ asciinema(name="secrets.cast") }}
+
+- **Remote Install:** Set up and manage Clan systems anywhere in the world with just a QR scan or SSH access, making remote installations as easy as snapping a photo or sharing a link.
+
+{{ asciinema(name="nixos-install.cast") }}
+
+
+## Who Stands to Benefit?
+
+Clan is for anyone and everyone who believes in the power of open source technology to connect, empower, and protect. From system administrators to less tech-savvy individuals, small business owners to privacy-conscious users, Clan offers something for everyone — a way to reclaim control and redefine how we interact with technology.
+
+## Join the Revolution
+
+Ready to control your digital world? Clan is more than a tool—it's a movement. Secure your data, manage your systems easily, or connect with others how you like. Start with Clan for a better digital future.
+
+Connect with us on our [Matrix channel at clan.lol](https://matrix.to/#/#clan:lassul.us) or through our IRC bridges (coming soon).
+
+Want to see the code? Check it out [on our Gitea](https://git.clan.lol/clan/clan-core) or [on GitHub](https://github.com/clan-lol/clan-core).
+
+Or follow our [RSS feed](https://docs.clan.lol/feed_rss_created.xml)!
+
+Join us and be part of changing technology for the better, together.
+
+

docs/site/blog/posts/new-docs.md

@@ -0,0 +1,13 @@
+---
+title: "New documentation site and weekly new meetup"
+authors:
+  - Lassulus
+  - Mic92
+date: 2024-04-16
+---
+
+Last week, we added a new documentation hub for clan at [docs.clan.lol](https://docs.clan.lol).
+We are still working on improving the installation procedures, so stay tuned.
+We now have weekly office hours where people are invited to hangout and ask questions.
+They are every Wednesday 15:30 UTC (17:30 CEST) in our [jitsi](https://jitsi.lassul.us/clan.lol).
+Otherwise drop by in our [matrix channel](https://matrix.to/#/#clan:lassul.us).

docs/site/contributing/contributing.md

@@ -0,0 +1 @@
+../../CONTRIBUTING.md

docs/site/drafts/install-iso.md

@@ -0,0 +1,146 @@
+# Hardware Installation
+
+For installations on physical hardware, create a NixOS installer image and transfer it to a bootable USB drive as described below.
+
+## Creating a Bootable USB Drive on Linux
+
+To create a bootable USB flash drive with the NixOS installer:
+
+### Download the install iso
+
+Either with wget:
+
+```shellSession
+wget https://github.com/nix-community/nixos-images/releases/download/nixos-unstable/nixos-installer-x86_64-linux.iso
+```
+
+or with curl:
+
+```shellSession
+curl -L https://github.com/nix-community/nixos-images/releases/download/nixos-unstable/nixos-installer-x86_64-linux.iso -o nixos-installer-x86_64-linux.iso
+```
+
+### Prepare the USB Flash Drive
+
+1. Insert your USB flash drive into your computer.
+
+2. Identify your flash drive with `lsblk`.
+
+    ```shellSession
+    lsblk
+    NAME                                          MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
+    sdb                                             8:0    1 117,2G  0 disk
+    └─sdb1                                          8:1    1 117,2G  0 part  /run/media/qubasa/INTENSO
+    nvme0n1                                       259:0    0   1,8T  0 disk
+    ├─nvme0n1p1                                   259:1    0   512M  0 part  /boot
+    └─nvme0n1p2                                   259:2    0   1,8T  0 part
+      └─luks-f7600028-9d83-4967-84bc-dd2f498bc486 254:0    0   1,8T  0 crypt /nix/store
+    ```
+
+    In this case it's `sdb`
+
+3. Ensure all partitions on the drive are unmounted. Replace `sdX` in the command below with your device identifier (like `sdb`, etc.):
+
+    ```shellSession
+    sudo umount /dev/sdb1
+    ```
+
+### Write the Image to the USB Drive
+
+Use the `dd` utility to write the NixOS installer image to your USB drive:
+
+  ```shellSession
+  sudo dd bs=4M conv=fsync oflag=direct status=progress if=./nixos-installer-x86_64-linux.iso of=/dev/sd<X>
+  ```
+
+  In this case, the USB device is `sdb` use `of=/dev/sdb`
+
+### Boot and Connect
+
+After writing the installer to the USB drive, use it to boot the target machine.
+
+1. For this secure boot needs to be disabled. Go into your UEFI / Bios settings by pressing one of the keys outlined below while booting:
+
+   - **Dell**: F2/Del (BIOS Setup)
+   - **HP**: Esc (Startup Menu)
+   - **Lenovo**: F2/Fn+F2/Novo Button (IdeaPad Boot Menu/BIOS Setup)
+   - **Acer**: F2/Del (BIOS Setup)
+   - **Asus**: F2/Del (BIOS Setup)
+   - **Toshiba**: Esc then F12 (Alternate Method)
+   - **Sony**: F11
+   - **Samsung**: F2 (BIOS Setup)
+   - **MSI**: Del (BIOS Setup)
+   - **Apple**: Option (Alt) Key (Boot Menu for Mac)
+   - If your hardware was not listed read the manufacturers instructions how to enter the boot Menu/BIOS Setup.
+
+2. Inside the UEFI/Bios Menu go to `Security->Secure Boot` and disable secure boot
+
+3. Save your settings. Put in the USB stick and reboot.
+
+4. Press one of keys outlined below to go into the Boot Menu
+
+       - **Dell**: F12 (Boot Menu)
+       - **HP**: F9 (Boot Menu)
+       - **Lenovo**: F12 (ThinkPad Boot Menu)
+       - **Acer**: F12 (Boot Menu)
+       - **Asus**: F8/Esc (Boot Menu)
+       - **Toshiba**: F12/F2 (Boot Menu)
+       - **Sony**: F11
+       - **Samsung**: F2/F12/Esc (Boot Menu)
+       - **MSI**: F11
+       - **Apple**: Option (Alt) Key (Boot Menu for Mac)
+       - If your hardware was not listed read the manufacturers instructions how to enter the boot Menu/BIOS Setup.
+
+
+5. Select `NixOS` to boot into the clan installer
+
+6. The installer will display an IP address and a root password, which you can use to connect via SSH.  
+    Alternatively you can also use the displayed QR code.
+
+7. Set your keyboard language (i.e. `de` for German keyboards, default is English). Important for writing passwords correctly.
+
+    ```shellSession
+    loadkeys de
+    ```
+
+8. If you only have Wifi available, execute:
+
+    1. Bring up the `iwd` shell
+
+        ```shellSession
+        iwctl
+        ```
+
+    2. List available networks. Double press tab after station for autocompleting your wlan device. In this case `wlan0`
+
+        ```shellSession
+        [iwd] station wlan0 get-networks
+        ```
+
+    3. Connect to a Wifi network. Replace `SSID` with the wlan network name.
+
+        ```shellSession
+        [iwd] station wlan0 connect SSID
+        ```
+
+9. Now that you have internet re-execute the init script by pressing `Ctrl+D` or by executing:
+
+    ```shellSession
+    bash
+    ```
+
+10. Connect to the machine over ssh
+
+    ```shellSession
+    ssh-copy-id -o PreferredAuthentications=password root@<ip>
+    ```
+
+    Use the root password displayed on your screen as login.
+
+---
+
+# Whats next?
+
+- Deploy a clan machine-configuration on your prepared machine
+
+---

docs/site/drafts/secret-cli.md

@@ -0,0 +1,324 @@
+## Secrets (CLI Reference)
+
+#### Adding Secrets (set)
+
+```bash
+clan secrets set mysecret
+> Paste your secret: 
+```
+
+!!! note
+    As you type your secret won't be displayed. Press Enter to save the secret.
+
+#### List all Secrets (list)
+
+```bash
+clan secrets list
+```
+
+#### Assigning Access (set)
+
+By default, secrets are encrypted for your key. To specify which users and machines can access a secret:
+
+```bash
+clan secrets set --machine <machine1> --machine <machine2> --user <user1> --user <user2> <secret_name>
+```
+
+#### Displaying Secrets (get)
+
+```bash
+clan secrets get mysecret
+```
+
+#### Rename
+
+TODO
+
+#### Remove
+
+TODO
+
+#### import-sops
+
+TODO
+
+### Users (Reference)
+
+Learn how to manage users and allowing access to existing secrets.
+
+#### list user
+
+Lists all added users
+
+```bash
+clan secrets user list
+```
+
+``` {.console, title="Example output", .no-copy}
+jon
+sara
+```
+
+!!! Question "Who can execute this command?"
+    Everyone - completely public.
+
+#### add user
+
+add a user
+
+```bash
+clan secrets users add {username} {public-key}
+```
+
+!!! Note
+    Changes can be trusted by maintainer review in version control.
+
+#### get user
+
+get a user public key
+
+```bash
+clan secrets users get {username}
+```
+
+``` {.console, title="Example output", .no-copy}
+age1zk8uzrte55wkg9lkqxu5x6twsj2ja4lehegks0cw4mkg6jv37d9qsjpt44
+```
+
+#### remove user
+
+remove a user
+
+```bash
+clan secrets users remove {username}
+```
+
+!!! Note
+    Changes can be trusted by maintainer review in version control.
+
+#### add-secret user
+
+Grants the user (`username`) access to the secret (`secret_name`)
+
+```bash
+clan secrets users add-secret {username} {secret_name}
+```
+
+!!! Note
+    Requires the executor of the command to have access to the secret (`secret_name`).
+
+#### remove-secret user
+
+remove the user (`username`) from accessing the secret (`secret_name`)
+
+!!! Danger "Make sure at least one person has access."
+    It might still be possible for the machine to access the secret. (See [machines](#machines))
+
+    We highly recommend to use version control such as `git` which allows you to rollback secrets in case anything gets messed up.
+
+```bash
+clan secrets users remove-secret {username} {secret_name}
+```
+
+!!! Question "Who can execute this command?"
+    Requires the executor of the command to have access to the secret (`secret_name`).
+
+### Machines (Reference)
+
+- [list](): list machines
+- [add](): add a machine
+- [get](): get a machine public key
+- [remove](): remove a machine
+- [add-secret](): allow a machine to access a secret
+- [remove-secret](): remove a machine's access to a secret
+
+#### List machine
+
+New machines in Clan come with age keys stored in `./sops/machines/<machine_name>`. To list these machines:
+
+```bash
+clan secrets machines list
+```
+
+#### Add machine
+
+For clan machines the machine key is generated automatically on demand if none exists.
+
+```bash
+clan secrets machines add <machine_name> <age_key>
+```
+
+If you already have a device key and want to add it manually, see: [How to obtain a remote key](#obtain-remote-keys-manually)
+
+#### get machine
+
+TODO
+
+#### remove machine
+
+TODO
+
+#### add-secret machine
+
+TODO
+
+#### remove-secret machine
+
+TODO
+
+### Groups (Reference)
+
+The Clan-CLI makes it easy to manage access by allowing you to create groups.
+
+- [list](): list groups
+- [add-user](): add a user to group
+- [remove-user](): remove a user from group
+- [add-machine](): add a machine to group
+- [remove-machine](): remove a machine from group
+- [add-secret](): allow a user to access a secret
+- [remove-secret](): remove a group's access to a secret
+
+#### List Groups
+
+```bash
+clan secrets groups list
+```
+
+#### add-user
+
+Assign users to a new group, e.g., `admins`:
+
+```bash
+clan secrets groups add-user admins <username>
+```
+
+!!! info
+    The group is created if no such group existed before.
+
+    The user must exist in beforehand (See: [users](#users-reference))
+
+    ```{.console, .no-copy}
+    .
+    ├── flake.nix
+    .   ...
+    └── sops
+        ├── groups
+        │   └── admins
+        │       └── users
+        │           └── <username> -> ../../../users/<username>
+    ```
+
+#### remove-user
+
+TODO
+
+#### add-machine
+
+TODO
+
+#### remove-machine
+
+TODO
+
+#### add-secret
+
+```bash
+clan secrets groups add-secret <group_name> <secret_name>
+```
+
+#### remove-secret
+
+TODO
+
+### Key (Reference)
+
+- [generate]() generate age key
+- [show]() show age public key
+- [update]() re-encrypt all secrets with current keys (useful when changing keys)
+
+#### generate
+
+TODO
+
+#### show
+
+TODO
+
+#### update
+
+TODO
+
+## Further
+
+Secrets in the repository follow this structure:
+
+```{.console, .no-copy}
+sops/
+├── secrets/
+│   └── <secret_name>/
+│       ├── secret
+│       └── users/
+│           └── <your_username>/
+```
+
+The content of the secret is stored encrypted inside the `secret` file under `mysecret`.
+
+By default, secrets are encrypted with your key to ensure readability.
+
+### Obtain remote keys manually
+
+To fetch a **SSH host key** from a preinstalled system:
+
+```bash
+ssh-keyscan <domain_name> | nix shell nixpkgs#ssh-to-age -c ssh-to-age
+```
+
+!!! Success
+    This command converts the SSH key into an age key on the fly. Since this is the format used by the clan secrets backend.
+
+    Once added the **SSH host key** enables seamless integration of existing machines with clan.
+
+Then add the key by executing:
+
+```bash
+clan secrets machines add <machine_name> <age_key>
+```
+
+See also: [Machine reference](#machines-reference)
+
+### NixOS integration
+
+A NixOS machine will automatically import all secrets that are encrypted for the
+current machine. At runtime it will use the host key to decrypt all secrets into
+an in-memory, non-persistent filesystem using [sops-nix](https://github.com/Mic92/sops-nix). 
+In your nixos configuration you can get a path to secrets like this `config.sops.secrets.<name>.path`. For example:
+
+```nix
+{ config, ...}: {
+  sops.secrets.my-password.neededForUsers = true;
+
+  users.users.mic92 = {
+    isNormalUser = true;
+    passwordFile = config.sops.secrets.my-password.path;
+  };
+}
+```
+
+See the [readme](https://github.com/Mic92/sops-nix) of sops-nix for more
+examples.
+
+### Migration: Importing existing sops-based keys / sops-nix
+
+`clan secrets` stores each secret in a single file, whereas [sops](https://github.com/Mic92/sops-nix) commonly allows to put all secrets in a yaml or json document.
+
+If you already happened to use sops-nix, you can migrate by using the `clan secrets import-sops` command by importing these files:
+
+```bash
+% clan secrets import-sops --prefix matchbox- --group admins --machine matchbox nixos/matchbox/secrets/secrets.yaml
+```
+
+This will create secrets for each secret found in `nixos/matchbox/secrets/secrets.yaml` in a `./sops` folder of your repository.
+Each member of the group `admins` in this case will be able to decrypt the secrets with their respective key.
+
+Since our clan secret module will auto-import secrets that are encrypted for a particular nixos machine,
+you can now remove `sops.secrets.<secrets> = { };` unless you need to specify more options for the secret like owner/group of the secret file.

docs/site/getting-started/backups.md

@@ -0,0 +1,149 @@
+# Backups
+
+## Introduction to Backups
+
+When you're managing your own services, creating regular backups is crucial to ensure your data's safety.
+This guide introduces you to Clan's built-in backup functionalities.
+Clan supports backing up your data to both local storage devices (like USB drives) and remote servers, using well-known tools like borgbackup and rsnapshot.
+We might add more options in the future, but for now, let's dive into how you can secure your data.
+
+## Backing Up Locally with Localbackup
+
+### What is Localbackup?
+
+Localbackup lets you backup your data onto physical storage devices connected to your computer,
+such as USB hard drives or network-attached storage. It uses a tool called rsnapshot for this purpose.
+
+### Setting Up Localbackup
+
+1. **Identify Your Backup Device:**
+
+First, figure out which device you'll use for backups. You can see all connected devices by running this command in your terminal:
+
+```bash
+lsblk --output NAME,PTUUID,FSTYPE,SIZE,MOUNTPOINT
+```
+
+Look for the device you intend to use for backups and note its details.
+
+2. **Configure Your Backup Device:**
+
+Once you've identified your device, you'll need to add it to your configuration.
+Here's an example NixOS configuration for a device located at `/dev/sda2` with an `ext4` filesystem:
+
+```nix
+{
+  fileSystems."/mnt/hdd" = {
+    device = "/dev/sda2";
+    fsType = "ext4";
+    options = [ "defaults" "noauto" ];
+  };
+}
+```
+
+Replace `/dev/sda2` with your device and `/mnt/hdd` with your preferred mount point.
+
+3. **Set Backup Targets:** Next, define where on your device you'd like the backups to be stored:
+
+   ```nix
+   {
+     clan.localbackup.targets.hdd = {
+       directory = "/mnt/hdd/backup";
+       mountpoint = "/mnt/hdd";
+     };
+   }
+   ```
+
+   Change `/mnt/hdd` to the actual mount point you're using.
+
+4. **Create Backups:** To create a backup, run:
+
+   ```bash
+   clan backups create mymachine
+   ```
+
+   This command saves snapshots of your data onto the backup device.
+
+5. **Listing Backups:** To see available backups, run:
+
+  ```bash
+  clan backups list mymachine
+  ```
+
+## Remote Backups with Borgbackup
+
+### Overview of Borgbackup
+
+Borgbackup splits the backup process into two parts: a backup client that sends data to a backup server.
+The server stores the backups.
+
+### Setting Up the Borgbackup Client
+
+1. **Specify Backup Server:**
+
+Start by indicating where your backup data should be sent. Replace `hostname` with your server's address:
+
+```nix
+{
+  clan.borgbackup.destinations = {
+    myhostname = {
+      repo = "borg@backuphost:/var/lib/borgbackup/myhostname";
+    };
+  };
+}
+```
+
+2. **Select Folders to Backup:**
+
+Decide which folders you want to back up. For example, to backup your home and root directories:
+
+```nix
+{ clanCore.state.userdata.folders = [ "/home" "/root" ]; }
+```
+
+3. **Generate Backup Credentials:**
+
+Run `clan facts generate <yourmachine>` to prepare your machine for backup, creating necessary SSH keys and credentials.
+
+### Setting Up the Borgbackup Server
+
+1. **Configure Backup Repository:**
+
+On the server where backups will be stored, enable the SSH daemon and set up a repository for each client:
+
+```nix
+{
+  services.borgbackup.repos.myhostname = {
+    path = "/var/lib/borgbackup/myhostname";
+    authorizedKeys = [
+      (builtins.readFile ./machines/myhostname/facts/borgbackup.ssh.pub)
+    ];
+  };
+}
+```
+
+Ensure the path to the public key is correct.
+
+2. **Update Your Systems:** Apply your changes by running `clan machines update` to both the server and your client
+
+### Managing Backups
+
+- **Scheduled Backups:**
+
+  Backups are automatically performed nightly. To check the next scheduled backup, use:
+
+  ```bash
+  systemctl list-timers | grep -E 'NEXT|borg'
+  ```
+
+- **Listing Backups:** To see available backups, run:
+
+  ```bash
+  clan backups list mymachine
+  ```
+
+- **Manual Backups:** You can also initiate a backup manually:
+
+  ```bash
+  clan backups create mymachine
+  ```

docs/site/getting-started/configure.md

@@ -0,0 +1,187 @@
+# Configuration - How to configure clan with your own machines
+
+## Global configuration
+
+In the `flake.nix` file:
+
+- [x] set a unique `clanName`.
+
+=== "**buildClan**"
+
+    ```nix title="clan-core.lib.buildClan"
+    buildClan {
+        # Set a unique name 
+        clanName = "Lobsters";
+        # Should usually point to the directory of flake.nix
+        directory = ./.;
+
+        machines = {
+            jon = {
+                # ...
+            };
+            # ...
+        }
+    }
+    ```
+
+=== "**flakeParts**"
+
+    !!! info "See [Clan with flake-parts](./flake-parts.md) for help migrating to flake-parts."
+
+    ```nix title="clan-core.flakeModules.default"
+    clan = {
+        # Set a unique name 
+        clanName = "Lobsters";
+
+        machines = {
+            jon = {
+                # ...
+            };
+            # ...
+        }
+    };
+    ```
+
+## Machine configuration
+
+Adding or configuring a new machine requires two simple steps:
+
+### Step 1. Identify Target Disk-ID
+
+1. Find the remote disk id by executing:
+
+    ```bash title="setup computer"
+    ssh root@flash-installer.local lsblk --output NAME,ID-LINK,FSTYPE,SIZE,MOUNTPOINT
+    ```
+
+    !!! Note
+        Replace `flash-installer.local` with the IP address of the machine if you don't have the avahi service running which resolves mDNS local domains.
+
+    Which should show something like:
+
+    ```{.shellSession hl_lines="6" .no-copy}
+    NAME        ID-LINK                                         FSTYPE   SIZE MOUNTPOINT
+    sda         usb-ST_16GB_AA6271026J1000000509-0:0                    14.9G 
+    ├─sda1      usb-ST_16GB_AA6271026J1000000509-0:0-part1                 1M 
+    ├─sda2      usb-ST_16GB_AA6271026J1000000509-0:0-part2      vfat     100M /boot
+    └─sda3      usb-ST_16GB_AA6271026J1000000509-0:0-part3      ext4     2.9G /
+    nvme0n1     nvme-eui.e8238fa6bf530001001b448b4aec2929              476.9G 
+    ├─nvme0n1p1 nvme-eui.e8238fa6bf530001001b448b4aec2929-part1 vfat     512M 
+    ├─nvme0n1p2 nvme-eui.e8238fa6bf530001001b448b4aec2929-part2 ext4   459.6G 
+    └─nvme0n1p3 nvme-eui.e8238fa6bf530001001b448b4aec2929-part3 swap    16.8G
+    ```
+
+1. Edit the following fields inside the `flake.nix`
+
+    === "**buildClan**"
+
+        ```nix title="clan-core.lib.buildClan" hl_lines="18 23"
+        buildClan {
+          # ...
+          machines = {
+            "jon" = {
+              imports = [
+                # ...
+                ./modules/disko.nix
+                ./machines/jon/configuration.nix
+              ];
+              # ...
+
+              # Change this to the correct ip-address or hostname
+              # The hostname is the machine name by default
+              clan.networking.targetHost = pkgs.lib.mkDefault "root@jon"
+
+              # Change this to the ID-LINK of the desired disk shown by 'lsblk'
+              disko.devices.disk.main = {
+                device = "/dev/disk/by-id/__CHANGE_ME__";
+              }
+
+              # e.g. > cat ~/.ssh/id_ed25519.pub
+              users.users.root.openssh.authorizedKeys.keys = [
+                  "<YOUR SSH_KEY>"
+              ];
+              # ...
+            };
+          };
+        }
+        ```
+
+    === "**flakeParts**"
+
+        ```nix title="clan-core.flakeModules.default" hl_lines="18 23"
+        clan = {
+          # ...
+          machines = {
+            "jon" = {
+              imports = [
+                # ...
+                ./modules/disko.nix
+                ./machines/jon/configuration.nix
+              ];
+              # ...
+
+              # Change this to the correct ip-address or hostname
+              # The hostname is the machine name by default
+              clan.networking.targetHost = pkgs.lib.mkDefault "root@jon"
+
+              # Change this to the ID-LINK of the desired disk shown by 'lsblk'
+              disko.devices.disk.main = {
+                device = "/dev/disk/by-id/__CHANGE_ME__";
+              }
+
+              # e.g. > cat ~/.ssh/id_ed25519.pub
+              users.users.root.openssh.authorizedKeys.keys = [
+                  "__YOUR_SSH_KEY__"
+              ];
+              # ...
+            };
+          };
+        };
+        ```
+
+
+!!! Info "Replace `__CHANGE_ME__` with the appropriate identifier, such as `nvme-eui.e8238fa6bf530001001b448b4aec2929`"
+!!! Info "Replace `__YOUR_SSH_KEY__` with your personal key, like `ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILoMI0NC5eT9pHlQExrvR5ASV3iW9+BXwhfchq0smXUJ jon@jon-desktop`"
+
+These steps will allow you to update your machine later.
+
+### Step 2: Detect Drivers
+
+Generate the `hardware-configuration.nix` file for your machine by executing the following command:
+
+```bash
+ssh root@flash-installer.local nixos-generate-config --no-filesystems --show-hardware-config > machines/jon/hardware-configuration.nix
+```
+
+This command connects to `flash-installer.local` as `root`, runs `nixos-generate-config` to detect hardware configurations (excluding filesystems), and writes them to `machines/jon/hardware-configuration.nix`.
+
+### Step 3: Custom Disk Formatting
+
+In `./modules/disko.nix`, a simple `ext4` disk partitioning scheme is defined for the Disko module. For more complex disk partitioning setups, refer to the [Disko examples](https://github.com/nix-community/disko/tree/master/example).
+
+### Step 4: Custom Configuration
+
+Modify `./machines/jon/configuration.nix` to personalize the system settings according to your requirements.
+
+### Step 5: Check Configuration
+
+Validate your configuration by running:
+
+```bash
+nix flake check
+```
+
+This command helps ensure that your system configuration is correct and free from errors.
+
+!!! Note
+
+    Integrate this step into your [Continuous Integration](https://en.wikipedia.org/wiki/Continuous_integration) workflow to ensure that only valid Nix configurations are merged into your codebase. This practice helps maintain system stability and reduces integration issues.
+
+
+---
+
+## Whats next?
+
+- [Secrets & Facts](secrets.md): Setting up secrets with nix-sops
+
+---