yadunut/clan-core
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

docs: clean up directory structure

Browse Source
This commit is contained in:
Johannes Kirschbauer
2024-04-13 14:42:37 +02:00
parent 4b152ddd5e
commit 77921f9c79
23 changed files with 10 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

149
docs/site/getting-started/backups.md Normal file
View File

@@ -0,0 +1,149 @@
# Backups
## Introduction to Backups
When you're managing your own services, creating regular backups is crucial to ensure your data's safety.
This guide introduces you to Clan's built-in backup functionalities.
Clan supports backing up your data to both local storage devices (like USB drives) and remote servers, using well-known tools like borgbackup and rsnapshot.
We might add more options in the future, but for now, let's dive into how you can secure your data.
## Backing Up Locally with Localbackup
### What is Localbackup?
Localbackup lets you backup your data onto physical storage devices connected to your computer,
such as USB hard drives or network-attached storage. It uses a tool called rsnapshot for this purpose.
### Setting Up Localbackup
1. **Identify Your Backup Device:**
First, figure out which device you'll use for backups. You can see all connected devices by running this command in your terminal:
```bash
lsblk --output NAME,PTUUID,FSTYPE,SIZE,MOUNTPOINT
```
Look for the device you intend to use for backups and note its details.
2. **Configure Your Backup Device:**
Once you've identified your device, you'll need to add it to your configuration.
Here's an example NixOS configuration for a device located at `/dev/sda2` with an `ext4` filesystem:
```nix
{
fileSystems."/mnt/hdd" = {
device = "/dev/sda2";
fsType = "ext4";
options = [ "defaults" "noauto" ];
};
}
```
Replace `/dev/sda2` with your device and `/mnt/hdd` with your preferred mount point.
3. **Set Backup Targets:** Next, define where on your device you'd like the backups to be stored:
```nix
{
clan.localbackup.targets.hdd = {
directory = "/mnt/hdd/backup";
mountpoint = "/mnt/hdd";
};
}
```
Change `/mnt/hdd` to the actual mount point you're using.
4. **Create Backups:** To create a backup, run:
```bash
clan backups create mymachine
```
This command saves snapshots of your data onto the backup device.
5. **Listing Backups:** To see available backups, run:
```bash
clan backups list mymachine
```
## Remote Backups with Borgbackup
### Overview of Borgbackup
Borgbackup splits the backup process into two parts: a backup client that sends data to a backup server.
The server stores the backups.
### Setting Up the Borgbackup Client
1. **Specify Backup Server:**
Start by indicating where your backup data should be sent. Replace `hostname` with your server's address:
```nix
{
clan.borgbackup.destinations = {
myhostname = {
repo = "borg@backuphost:/var/lib/borgbackup/myhostname";
};
};
}
```
2. **Select Folders to Backup:**
Decide which folders you want to back up. For example, to backup your home and root directories:
```nix
{ clanCore.state.userdata.folders = [ "/home" "/root" ]; }
```
3. **Generate Backup Credentials:**
Run `clan facts generate <yourmachine>` to prepare your machine for backup, creating necessary SSH keys and credentials.
### Setting Up the Borgbackup Server
1. **Configure Backup Repository:**
On the server where backups will be stored, enable the SSH daemon and set up a repository for each client:
```nix
{
services.borgbackup.repos.myhostname = {
path = "/var/lib/borgbackup/myhostname";
authorizedKeys = [
(builtins.readFile ./machines/myhostname/facts/borgbackup.ssh.pub)
];
};
}
```
Ensure the path to the public key is correct.
2. **Update Your Systems:** Apply your changes by running `clan machines update` to both the server and your client
### Managing Backups
- **Scheduled Backups:**
Backups are automatically performed nightly. To check the next scheduled backup, use:
```bash
systemctl list-timers | grep -E 'NEXT|borg'
```
- **Listing Backups:** To see available backups, run:
```bash
clan backups list mymachine
```
- **Manual Backups:** You can also initiate a backup manually:
```bash
clan backups create mymachine
```

93
docs/site/getting-started/flake-parts.md Normal file
View File

@@ -0,0 +1,93 @@
# Clan with `flake-parts`
Clan supports integration with [flake.parts](https://flake.parts/) a tool which allows modular compositions.
Here's how to set up Clan using flakes and flake-parts.
## 1. Update Your Flake Inputs
To begin, you'll need to add `flake-parts` as a new dependency in your flake's inputs. This is alongside the already existing dependencies, such as `flake-parts` and `nixpkgs`. Here's how you can update your `flake.nix` file:
```nix
# flake.nix
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
# New flake-parts input
flake-parts.url = "github:hercules-ci/flake-parts";
flake-parts.inputs.nixpkgs-lib.follows = "nixpkgs";
clan-core = {
url = "git+https://git.clan.lol/clan/clan-core";
inputs.nixpkgs.follows = "nixpkgs"; # Needed if your configuration uses nixpkgs unstable.
# New
inputs.flake-parts.follows = "flake-parts";
};
}
```
## 2. Import Clan-Core Flake Module
After updating your flake inputs, the next step is to import the `clan-core` flake module. This will make the [clan options](https://git.clan.lol/clan/clan-core/src/branch/main/flakeModules/clan.nix) available within `mkFlake`.
```nix
outputs =
inputs@{ flake-parts, ... }:
flake-parts.lib.mkFlake { inherit inputs; } (
{
imports = [
inputs.clan-core.flakeModules.default
];
}
);
```
### 3. Configure Clan Settings and Define Machines
Configure your clan settings and define machine configurations.
Below is a guide on how to structure this in your flake.nix:
```nix
outputs =
inputs@{ flake-parts, ... }:
flake-parts.lib.mkFlake { inherit inputs; } (
{
imports = [
inputs.clan-core.flakeModules.default
];
clan = {
## Clan wide settings. (Required)
clanName = "__CHANGE_ME__"; # Ensure to choose a unique name.
directory = self; # Point this to the repository root.
specialArgs = { }; # Add arguments to every nix import in here
machines = {
jons-desktop = {
nixpkgs.hostPlatform = "x86_64-linux";
imports = [
clan-core.clanModules.sshd # Add openssh server for Clan management
./configuration.nix
];
};
};
};
}
);
```
For detailed information about configuring `flake-parts` and the available options within Clan,
refer to the Clan module documentation located [here](https://git.clan.lol/clan/clan-core/src/branch/main/flakeModules/clan.nix).
### Next Steps
With your flake created, explore how to add new machines by reviewing the documentation provided [here](machines.md).
---
## TODO
* How do I use Clan machines install to setup my current machine?
* I probably need the clan-core sshd module for that?
* We need to tell them that configuration.nix of a machine NEEDS to be under the directory CLAN_ROOT/machines/<machine-name> I think?

134
docs/site/getting-started/installer.md Normal file
View File

@@ -0,0 +1,134 @@
# Installer
We offer a dedicated installer to assist remote installations.
In this tutorial we will guide you through building and flashing it to a bootable USB drive.
## Step 0. Prerequisites
- [x] A free USB Drive with at least 1.5GB (All data on it will be lost)
- [x] Linux/NixOS Machine with Internet
## Step 1. Identify the USB Flash Drive
1. Insert your USB flash drive into your computer.
2. Identify your flash drive with `lsblk`
```bash
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sdb 8:0 1 117,2G 0 disk
└─sdb1 8:1 1 117,2G 0 part /run/media/qubasa/INTENSO
nvme0n1 259:0 0 1,8T 0 disk
├─nvme0n1p1 259:1 0 512M 0 part /boot
└─nvme0n1p2 259:2 0 1,8T 0 part
└─luks-f7600028-9d83-4967-84bc-dd2f498bc486 254:0 0 1,8T 0 crypt /nix/store /
```
In this case it's `sdb`
3. Ensure all partitions on the drive are unmounted. Replace `sdb1` in the command below with your device identifier (like `sdc1`, etc.):
```bash
sudo umount /dev/sdb1
```
### Step 2. Build the Installer
```bash
nix build git+https://git.clan.lol/clan/clan-core.git#install-iso
```
### Step 3. Flash the Installer to the USB Drive
Use the `dd` utility to write the NixOS installer image to your USB drive:
```bash
sudo dd bs=4M conv=fsync oflag=direct status=progress if=./result of=/dev/sd<X>
```
In case your USB device is `sdb` use `of=/dev/sdb`
### Step 4. Boot and Connect to your network
After writing the installer to the USB drive, use it to boot the target machine.
> i.e. Plug it into the target machine and select the USB drive as a temporary boot device.
For most hardware you can find the Key-combination below:
- **Dell**: F12 (Boot Menu), F2/Del (BIOS Setup)
- **HP**: F9 (Boot Menu), Esc (Startup Menu)
- **Lenovo**: F12 (ThinkPad Boot Menu), F2/Fn+F2/Novo Button (IdeaPad Boot Menu/BIOS Setup)
- **Acer**: F12 (Boot Menu), F2/Del (BIOS Setup)
- **Asus**: F8/Esc (Boot Menu), F2/Del (BIOS Setup)
- **Toshiba**: F12/F2 (Boot Menu), Esc then F12 (Alternate Method)
- **Sony**: F11/Assist Button (Boot Menu/Recovery Options)
- **Samsung**: F2/F12/Esc (Boot Menu), F2 (BIOS Setup)
- **MSI**: F11 (Boot Menu), Del (BIOS Setup)
- **Apple**: Option (Alt) Key (Boot Menu for Mac)
- If your hardware was not listed read the manufacturers instructions how to enter the boot Menu/BIOS Setup.
**During Boot**
Select `NixOS` to boot into the clan installer.
**After Booting**
For deploying your configuration the machine needs to be connected via LAN (recommended).
For connecting via Wifi, please consult the guide below.
---
### Whats next?
- [Deploying Machines](machines.md): Deploying a Machine configuration
- [WiFi](#optional-connect-to-wifi): Guide for connecting to Wifi.
---
### (Optional) Connect to Wifi
If you don't have access via LAN the Installer offers support for connecting via Wifi.
```bash
iwctl
```
This will enter `iwd`
```bash
[iwd]#
```
Now run the following command to connect to your Wifi:
```bash
# Identify your network device.
device list
# Replace 'wlan0' with your device name
# Find your Wifi SSID.
station wlan0 scan
station wlan0 get-networks
# Replace your_ssid with the Wifi SSID
# Connect to your network.
station wlan0 connect your_ssid
# Verify you are connected
station wlan0 show
```
If the connection was successful you should see something like this:
```bash
State connected
Connected network FRITZ!Box (Your router device)
IPv4 address 192.168.188.50 (Your new local ip)
```
Press `ctrl-d` to exit `IWD`
Press `ctrl-d` **again** to update the displayed QR code and connection information.

181
docs/site/getting-started/machines.md Normal file
View File

@@ -0,0 +1,181 @@
# Deploy Machine
Integrating a new machine into your Clan environment is an easy yet flexible process, allowing for a straight forward management of multiple NixOS configurations.
We'll walk you through adding a new computer to your Clan.
## Installing a New Machine
Clan CLI, in conjunction with [nixos-anywhere](https://github.com/nix-community/nixos-anywhere), provides a seamless method for installing NixOS on various machines.
This process involves preparing a suitable hardware and disk partitioning configuration and ensuring the target machine is accessible via SSH.
### Step 0. Prerequisites
- [x] **Two Computers**: You need one computer that you're getting ready (we'll call this the Target Computer) and another one to set it up from (we'll call this the Setup Computer). Make sure both can talk to each other over the network using SSH.
=== "**Physical Hardware**"
!!! How
1. Create a NixOS installer image and transfer it to a bootable USB drive as described in the [installer](./installer.md).
2. Boot the target machine and connect it to a network that makes it reachable from your setup computer.
=== "**Cloud Machines**"
!!! How
Use any linux machine if it is reachable via SSH and supports `kexec`.
Confirm the machine is reachable via SSH from your setup computer.
```bash
ssh root@<your_target_machine_ip>
```
- [x] **Machine configuration**: You want to deploy. [Check out our templates](../templates/index.md)
- [x] Initialized secrets: See [secrets](secrets.md) for how to initialize your secrets.
- [x] (Optional) USB Flash Drive with the [Clan Installer](installer.md)
### Step 1. Identify Target Disk-ID
```bash
lsblk --output NAME,ID-LINK,FSTYPE,SIZE,MOUNTPOINT
# Should print something like:
NAME ID-LINK FSTYPE SIZE MOUNTPOINT
sda usb-ST_16GB_AA6271026J1000000509-0:0 14.9G
├─sda1 usb-ST_16GB_AA6271026J1000000509-0:0-part1 1M
├─sda2 usb-ST_16GB_AA6271026J1000000509-0:0-part2 vfat 100M /boot
└─sda3 usb-ST_16GB_AA6271026J1000000509-0:0-part3 ext4 2.9G /
nvme0n1 nvme-eui.e8238fa6bf530001001b448b4aec2929 476.9G
├─nvme0n1p1 nvme-eui.e8238fa6bf530001001b448b4aec2929-part1 vfat 512M
├─nvme0n1p2 nvme-eui.e8238fa6bf530001001b448b4aec2929-part2 ext4 459.6G
└─nvme0n1p3 nvme-eui.e8238fa6bf530001001b448b4aec2929-part3 swap 16.8G
```
Now change the following lines of your configuration you want to deploy.
We need to set the hardware specific `disk-id` (i.e. `nvme-eui.e8238fa6bf530001001b448b4aec2929`)
Import the clan `diskLayouts` Module.
```
imports = [
clan-core.clanModules.diskLayouts
]
```
```nix
# flake.nix / configuration.nix
clan.diskLayouts.singleDiskExt4 = {
device = "/dev/disk/by-id/<MY_DISK_ID>";
}
```
Also set the targetHost: (i.e. user `root` hostname `jon`)
The hostname is the **machine name** by default
```nix
clan.networking.targetHost = pkgs.lib.mkDefault "root@jon"
```
`cd` into your `my-clan` directory
```bash
my-clan (main)> tree
.
├── flake.lock
├── flake.nix
└── machines
└── jon
└── configuration.nix
```
And verify that the machine configuration is detected from the `clan` cli.
```bash
clan machines list
#> jon
```
### Step 3. Deploy the machine
**Finally deployment time!** Use the following command to build and deploy the image via SSH onto your machine.
Replace `<target_host>` with the **installer's ip address**:
```bash
clan machines install my-machine <target_host>
```
> Note: This may take a while for building and for the file transfer.
🎉 🚀 Your machine is all set up
---
## What's next ?
- [**Update a Machine**](#update-your-machines): Learn how to update an existing machine?
Coming Soon:
- **Join Your Machines in a Private Network:**: Stay tuned for steps on linking all your machines into a secure mesh network with Clan.
---
## Update Your Machines
Clan CLI enables you to remotely update your machines over SSH. This requires setting up a target address for each target machine.
### Setting the Target Host
Replace `host_or_ip` with the actual hostname or IP address of your target machine:
```bash
clan config --machine my-machine clan.networking.targetHost root@host_or_ip
```
> Note: The use of `root@` in the target address implies SSH access as the `root` user.
> Ensure that the root login is secured and only used when necessary.
### Updating Machine Configurations
Execute the following command to update the specified machine:
```bash
clan machines update my-machine
```
You can also update all configured machines simultaneously by omitting the machine name:
```bash
clan machines update
```
### Setting a Build Host
If the machine does not have enough resources to run the NixOS evaluation or build itself,
it is also possible to specify a build host instead.
During an update, the cli will ssh into the build host and run `nixos-rebuild` from there.
```bash
clan config --machine my-machine clan.networking.buildHost root@host_or_ip
```
### Excluding a machine from `clan machine update`
To exclude machines from being updated when running `clan machines update` without any machines specified,
one can set the `clan.deployment.requireExplicitUpdate` option to true:
```bash
clan config --machine my-machine clan.deployment.requireExplicitUpdate true
```
This is useful for machines that are not always online or are not part of the regular update cycle.
---
# TODO:
* TODO: How to join others people zerotier
* `services.zerotier.joinNetworks = [ "network-id" ]`
* Controller needs to approve over webinterface or cli

69
docs/site/getting-started/networking.md Normal file
View File

@@ -0,0 +1,69 @@
# Overlay Networks
This guide provides detailed instructions for configuring
[ZeroTier VPN](https://zerotier.com) within Clan. Follow the
outlined steps to set up a machine as a VPN controller (`<CONTROLLER>`) and to
include a new machine into the VPN.
## 1. Setting Up the VPN Controller
The VPN controller is initially essential for providing configuration to new
peers. Post the address allocation, the controller's continuous operation is not
crucial.
### Instructions:
1. **Designate a Machine**: Label a machine as the VPN controller in the clan,
referred to as `<CONTROLLER>` henceforth in this guide.
2. **Add Configuration**: Input the below configuration to the NixOS
configuration of the controller machine:
```nix
clan.networking.zerotier.controller = {
enable = true;
public = true;
};
```
3. **Update the Controller Machine**: Execute the following:
```bash
$ clan machines update <CONTROLLER>
```
Your machine is now operational as the VPN controller.
## 2. Integrating a New Machine to the VPN
To introduce a new machine to the VPN, adhere to the following steps:
### Instructions:
1. **Update Configuration**: On the new machine, incorporate the below to its
configuration, substituting `<CONTROLLER>` with the controller machine name:
```nix
{ config, ... }: {
clan.networking.zerotier.networkId = builtins.readFile (config.clanCore.clanDir + "/machines/<CONTROLLER>/facts/zerotier-network-id");
}
```
2. **Update the New Machine**: Execute:
```bash
$ clan machines update <NEW_MACHINE>
```
Replace `<NEW_MACHINE>` with the designated new machine name.
3. **Retrieve the ZeroTier ID**: On the `new_machine`, execute:
```bash
$ sudo zerotier-cli info
```
Example Output: `200 info d2c71971db 1.12.1 OFFLINE`, where `d2c71971db` is
the ZeroTier ID.
4. **Authorize the New Machine on Controller**: On the controller machine,
execute:
```bash
$ sudo zerotier-members allow <ID>
```
Substitute `<ID>` with the ZeroTier ID obtained previously.
5. **Verify Connection**: On the `new_machine`, re-execute:
```bash
$ sudo zerotier-cli info
```
The status should now be "ONLINE" e.g., `200 info 47303517ef 1.12.1 ONLINE`.
Congratulations! The new machine is now part of the VPN, and the ZeroTier
configuration on NixOS within the Clan project is complete.

193
docs/site/getting-started/secrets.md Normal file
View File

@@ -0,0 +1,193 @@
# Secrets
Clan enables encryption of secrets (such as passwords & keys) ensuring security and ease-of-use among users.
Clan utilizes the [sops](https://github.com/getsops/sops) format and integrates with [sops-nix](https://github.com/Mic92/sops-nix) on NixOS machines.
This documentation will guide you through managing secrets with the Clan CLI
## 1. Initializing Secrets
### Create Your Master Keypair
To get started, you'll need to create **Your master keypair**.
Don't worry — if you've already made one before, this step won't change or overwrite it.
```bash
clan secrets key generate
```
**Output**:
```bash
Public key: age1wkth7uhpkl555g40t8hjsysr20drq286netu8zptw50lmqz7j95sw2t3l7
Generated age private key at '/home/joerg/.config/sops/age/keys.txt' for your user. Please back it up on a secure location or you will lose access to your secrets.
Also add your age public key to the repository with 'clan secrets users add YOUR_USER age1wkth7uhpkl555g40t8hjsysr20drq286netu8zptw50lmqz7j95sw2t3l7' (replace YOUR_USER with your actual username)
```
⚠️ **Important**: Make sure to keep a safe backup of the private key you've just created.
If it's lost, you won't be able to get to your secrets anymore because they all need the master key to be unlocked.
> Note: It's safe to add any secrets created by the clan CLI and placed in your repository to version control systems like `git`.
### Add Your Public Key
```bash
clan secrets users add <your_username> <your_public_key>
```
⚠️ **Important**: Choose the username same username as on your Setup/Source Machine that you use to control the deployment with.
Once run this will create the following files:
```bash
sops/
└── users/
└── <your_username>/
└── key.json
```
## 2. Adding Machine Keys
New machines in Clan come with age keys stored in `./sops/machines/<machine_name>`. To list these machines:
```bash
$ clan secrets machines list
```
For existing machines, add their keys:
```bash
$ clan secrets machines add <machine_name> <age_key>
```
### Advanced
To fetch an age key from an SSH host key:
```bash
$ ssh-keyscan <domain_name> | nix shell nixpkgs#ssh-to-age -c ssh-to-age
```
## 3. Assigning Access
By default, secrets are encrypted for your key. To specify which users and machines can access a secret:
```bash
$ clan secrets set --machine <machine1> --machine <machine2> --user <user1> --user <user2> <secret_name>
```
You can add machines/users to existing secrets without modifying the secret:
```bash
$ clan secrets machines add-secret <machine_name> <secret_name>
```
## 4. Adding Secrets
```bash
$ clan secrets set mysecret
Paste your secret:
```
> Note: As you type - your secret won't be displayed. Press Enter to save the secret.
## 5. Retrieving Stored Secrets
```bash
$ clan secrets get mysecret
```
### List all Secrets
```bash
$ clan secrets list
```
## 6. Groups
Clan CLI makes it easy to manage access by allowing you to create groups.
All users within a group inherit access to all secrets of the group.
This feature eases the process of handling permissions for multiple users.
Here's how to get started:
1. **Creating Groups**:
Assign users to a new group, e.g., `admins`:
```bash
$ clan secrets groups add admins <username>
```
2. **Listing Groups**:
```bash
$ clan secrets groups list
```
3. **Assigning Secrets to Groups**:
```bash
$ clan secrets groups add-secret <group_name> <secret_name>
```
## Further
Secrets in the repository follow this structure:
```bash
sops/
├── secrets/
│ └── <secret_name>/
│ ├── secret
│ └── users/
│ └── <your_username>/
```
The content of the secret is stored encrypted inside the `secret` file under `mysecret`.
By default, secrets are encrypted with your key to ensure readability.
### NixOS integration
A NixOS machine will automatically import all secrets that are encrypted for the
current machine. At runtime it will use the host key to decrypt all secrets into
a in-memory, non-persistent filesystem using
[sops-nix](https://github.com/Mic92/sops-nix). In your nixos configuration you
can get a path to secrets like this `config.sops.secrets.<name>.path`. Example:
```nix
{ config, ...}: {
sops.secrets.my-password.neededForUsers = true;
users.users.mic92 = {
isNormalUser = true;
passwordFile = config.sops.secrets.my-password.path;
};
}
```
See the [readme](https://github.com/Mic92/sops-nix) of sops-nix for more
examples.
### Migration: Importing existing sops-based keys / sops-nix
`clan secrets` stores each secrets in a single file, whereas [sops](https://github.com/Mic92/sops-nix)
commonly allows to put all secrets in a yaml or json documents.
If you already happened to use sops-nix, you can migrate by using the `clan secrets import-sops` command by importing these documents:
```bash
% clan secrets import-sops --prefix matchbox- --group admins --machine matchbox nixos/matchbox/secrets/secrets.yaml
```
This will create secrets for each secret found in `nixos/matchbox/secrets/secrets.yaml` in a ./sops folder of your repository.
Each member of the group `admins` will be able
Since our clan secret module will auto-import secrets that are encrypted for a particular nixos machine,
you can now remove `sops.secrets.<secrets> = { };` unless you need to specify more options for the secret like owner/group of the secret file.