yadunut/clan-core
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge pull request 'docs(authoring): restructure authoring guides' (#3248) from hsjobeki/clan-core:docs-authoring into main

Browse Source 
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/3248
This commit is contained in:
hsjobeki
2025-04-08 19:58:59 +00:00
parent aa75cc4777 b4f63b8835
commit f60da91dea
7 changed files with 202 additions and 208 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
docs/mkdocs.yml
View File

@@ -58,20 +58,19 @@ nav:
- Autoincludes: manual/adding-machines.md - Autoincludes: manual/adding-machines.md
- Inventory: - Inventory:
- Inventory: manual/inventory.md - Inventory: manual/inventory.md
- Services: manual/distributed-services.md - Instances: manual/distributed-services.md
- Secure Boot: manual/secure-boot.md - Secure Boot: manual/secure-boot.md
- Flake-parts: manual/flake-parts.md - Flake-parts: manual/flake-parts.md
- Authoring: - Authoring:
- Modules: clanmodules/index.md - clan.service: authoring/clanServices/index.md
- Disk Templates: manual/disk-templates.md - Disk Templates: authoring/templates/disk/disko-templates.md
- clanModules: authoring/legacyModules/index.md
- Contributing: - Contributing:
- Contribute: contributing/contribute.md - Contribute: contributing/contribute.md
- Debugging: contributing/debugging.md - Debugging: contributing/debugging.md
- Testing: contributing/testing.md - Testing: contributing/testing.md
- Repo Layout: manual/repo-layout.md - Repo Layout: manual/repo-layout.md
- Migrate existing Flakes: manual/migration-guide.md - Migrate existing Flakes: manual/migration-guide.md
# - Concepts:
# - Overview: concepts/index.md
- Reference: - Reference:
- Overview: reference/index.md - Overview: reference/index.md
- Clan Modules: - Clan Modules:

186
docs/site/authoring/clanServices/index.md Normal file
View File

@@ -0,0 +1,186 @@
# Authoring a 'clan.service' module
!!! Tip
This is the successor format to the older [clanModules](../legacyModules/index.md)
While some features might still be missing we recommend to adapt this format early and give feedback.
## Service Module Specification
This section explains how to author a clan service module.
We discussed the initial architecture in [01-clan-service-modules](https://git.clan.lol/clan/clan-core/src/branch/main/decisions/01-ClanModules.md) and decided to rework the format as follows:
### A Minimal module
First of all we need to register our module into the `inventory.modules` attribute. Make sure to choose a unique name so the module doesn't have a name collision with any of the core modules.
While not required we recommend to prefix your module attribute name.
If you export the module from your flake, other people will be able to import it and use it within their clan.
i.e. `@hsjobeki/customNetworking`
```nix title=flake.nix
# ...
outputs = inputs: flake-parts.lib.mkFlake { inherit inputs; } ({
imports = [ inputs.clan-core.flakeModules.default ];
# ...
clan = {
inventory = {
# We could also inline the complete module spec here
# For example
# {...}: { _class = "clan.service"; ... };
modules."@hsjobeki/customNetworking" = import ./service-modules/networking.nix;
};
# If needed: Exporting the module for other people
modules."@hsjobeki/customNetworking" = import ./service-modules/networking.nix;
};
})
```
The imported module file must fulfill at least the following requirements:
- Be an actual module. That means: Be either an attribute set; or a function that returns an attribute set.
- Required `_class = "clan.service"
- Required `manifest.name = "<name of the provided service>"`
```nix title="/service-modules/networking.nix"
{
_class = "clan.service";
manifest.name = "zerotier-networking";
# ...
}
```
### Adding functionality to the module
While the very minimal module is valid in itself it has no way of adding any machines to it, because it doesn't specify any roles.
The next logical step is to think about the interactions between the machines and define *roles* for them.
Here is a short guide with some conventions:
- [ ] If they all have the same relation to each other `peer` is commonly used. `peers` can often talk to each other directly.
- [ ] Often machines don't necessarily have direct relation to each other and there is one elevated machine in the middle classically know as `client-server`. `clients` are less likely to talk directly to each other than `peers`
- [ ] If your machines don't have any relation and/or interactions to each other you should reconsider if the desired functionality is really a multi-host service.
```nix title="/service-modules/networking.nix"
{
_class = "clan.service";
manifest.name = "zerotier-networking";
# Define what roles exist
roles.peer = {};
roles.controller = {};
# ...
}
```
Next we need to define the settings and the behavior of these distinct roles.
```nix title="/service-modules/networking.nix"
{
_class = "clan.service";
manifest.name = "zerotier-networking";
# Define what roles exist
roles.peer = {
interface = {
# These options can be set via 'roles.client.settings'
options.ipRanges = mkOption { type = listOf str; };
};
# Maps over all instances and produces one result per instance.
perInstance = { instanceName, settings, machine, roles, ... }: {
# Analog to 'perSystem' of flake-parts.
# For every instance of this service we will add a nixosModule to a client-machine
nixosModule = { config, ... }: {
# Interaction examples what you could do here:
# - Get some settings of this machine
# settings.ipRanges
#
# - Get all controller names:
# allControllerNames = lib.attrNames roles.controller.machines
#
# - Get all roles of the machine:
# machine.roles
#
# - Get the settings that where applied to a specific controller machine:
# roles.controller.machines.jon.settings
#
# Add one systemd service for every instance
systemd.services.zerotier-client-${instanceName} = {
# ... depend on the '.config' and 'perInstance arguments'
};
};
}
};
roles.controller = {
interface = {
# These options can be set via 'roles.server.settings'
options.dynamicIp.enable = mkOption { type = bool; };
};
perInstance = { ... }: {};
};
# Maps over all machines and produces one result per machine.
perMachine = { instances, machine, ... }: {
# Analog to 'perSystem' of flake-parts.
# For every machine of this service we will add exactly one nixosModule to a machine
nixosModule = { config, ... }: {
# Interaction examples what you could do here:
# - Get the name of this machine
# machine.name
#
# - Get all roles of this machine across all instances:
# machine.roles
#
# - Get the settings of a specific instance of a specific machine
# instances.foo.roles.peer.machines.jon.settings
#
# Globally enable something
networking.enable = true;
};
};
# ...
}
```
## Using values from a NixOS machine inside the module
!!! Example "Experimental Status"
This feature is experimental and should be used with care.
Sometimes a settings value depends on something within a machines `config`.
Since the `interface` is defined completely machine-agnostic this means values from a machine cannot be set in the traditional way.
The following example shows how to create a local instance of machine specific settings.
```nix title="someservice.nix"
{
# Maps over all instances and produces one result per instance.
perInstance = { instanceName, extendSettings, machine, roles, ... }: {
nixosModule = { config, ... }:
let
# Create new settings with
# 'ipRanges' defaulting to 'config.network.ip.range' from this machine
# This only works if there is no 'default' already.
localSettings = extendSettings {
ipRanges = lib.mkDefault config.network.ip.range;
};
in
{
# ...
};
};
}
```
!!! Danger
`localSettings` are a local attribute. Other machines cannot access it.
If calling extendSettings is done that doesn't change the original `settings` this means if a different machine tries to access i.e `roles.client.settings` it would *NOT* contain your changes.
Exposing the changed settings to other machines would come with a huge performance penalty, thats why we don't want to offer it.

14
docs/site/clanmodules/index.md → docs/site/authoring/legacyModules/index.md
View File

@@ -1,19 +1,19 @@
# Authoring a clanModule # Authoring a clanModule
!!! Danger ":fontawesome-solid-road-barrier: Under Construction :fontawesome-solid-road-barrier:" !!! Danger "Will get deprecated soon"
Currently under construction use with caution Please consider twice creating new modules in this format
:fontawesome-solid-road-barrier: :fontawesome-solid-road-barrier: :fontawesome-solid-road-barrier: [`clan.service` module](../clanServices/index.md) will be the new standard soon.
This site will guide you through authoring your first module. Explaining which conventions must be followed, such that others will have an enjoyable experience and the module can be used with minimal effort. This site will guide you through authoring your first module. Explaining which conventions must be followed, such that others will have an enjoyable experience and the module can be used with minimal effort.
!!! Tip !!! Tip
External ClanModules can be ad-hoc loaded via [`clan.inventory.modules`](../reference/nix-api/inventory.md#inventory.modules) External ClanModules can be ad-hoc loaded via [`clan.inventory.modules`](../../reference/nix-api/inventory.md#inventory.modules)
## Bootstrapping the `clanModule` ## Bootstrapping the `clanModule`
A ClanModule is a specific subset of a [NixOS Module](https://nix.dev/tutorials/module-system/index.html), but it has some constraints and might be used via the [Inventory](../manual/inventory.md) interface. A ClanModule is a specific subset of a [NixOS Module](https://nix.dev/tutorials/module-system/index.html), but it has some constraints and might be used via the [Inventory](../../manual/inventory.md) interface.
In fact a `ClanModule` can be thought of as a layer of abstraction on-top of NixOS and/or other ClanModules. It may configure sane defaults and provide an ergonomic interface that is easy to use and can also be used via a UI that is under development currently. In fact a `ClanModule` can be thought of as a layer of abstraction on-top of NixOS and/or other ClanModules. It may configure sane defaults and provide an ergonomic interface that is easy to use and can also be used via a UI that is under development currently.
Because ClanModules should be configurable via `json`/`API` all of its interface (`options`) must be serializable. Because ClanModules should be configurable via `json`/`API` all of its interface (`options`) must be serializable.
@@ -48,7 +48,7 @@ clanModules/borgbackup
=== "User module" === "User module"
If the module should be ad-hoc loaded. If the module should be ad-hoc loaded.
It can be made available in any project via the [`clan.inventory.modules`](../reference/nix-api/inventory.md#inventory.modules) attribute. It can be made available in any project via the [`clan.inventory.modules`](../../reference/nix-api/inventory.md#inventory.modules) attribute.
```nix title="flake.nix" ```nix title="flake.nix"
# ... # ...
@@ -89,7 +89,7 @@ description = "Module A"
This is the example module that does xyz. This is the example module that does xyz.
``` ```
See the [Full Frontmatter reference](../reference/clanModules/frontmatter/index.md) further details and all supported attributes. See the [Full Frontmatter reference](../../reference/clanModules/frontmatter/index.md) further details and all supported attributes.
## Roles ## Roles

0
docs/site/manual/disk-templates.md → docs/site/authoring/templates/disk/disko-templates.md
View File

4
docs/site/index.md
View File

@@ -27,9 +27,9 @@ hide:
--- ---
Create clanModules that can be reused by the community. Create ressources that can be reused by the community.
[:octicons-arrow-right-24: Authoring clanModules](./clanmodules/index.md) [:octicons-arrow-right-24: Authoring guides](./authoring/legacyModules/index.md)
</div> </div>

195
docs/site/manual/distributed-services.md
View File

@@ -1,12 +1,7 @@
# Services # Instances
First of all it might be needed to explain what we mean by the term *distributed service* First of all it might be needed to explain what we mean by the term *distributed service*
!!! Note
Currently there are two ways of using such services.
1. via `inventory.services` **Will be deprecated**
2. via `inventory.instances` **Will be the new `inventory.services` once everyone has migrated**
## What is considered a service? ## What is considered a service?
A **distributed service** is a system where multiple machines work together to provide a certain functionality, abstracting complexity and allowing for declarative configuration and management. A **distributed service** is a system where multiple machines work together to provide a certain functionality, abstracting complexity and allowing for declarative configuration and management.
@@ -17,7 +12,7 @@ The term **Multi-host-service-abstractions** was introduced previously in the [n
## How to use such a Service in Clan? ## How to use such a Service in Clan?
In clan everyone can provide services via modules. Those modules must comply to a certain [specification](#service-module-specification), which is discussed later. In clan everyone can provide services via modules. Those modules must be [`clan.service` modules](../authoring/clanServices/index.md).
To use a service you need to create an instance of it via the `clan.inventory.instances` attribute: To use a service you need to create an instance of it via the `clan.inventory.instances` attribute:
The source of the module must be specified as a simple string. The source of the module must be specified as a simple string.
@@ -131,189 +126,3 @@ The following example shows how to use remote modules and configure them for use
); );
} }
``` ```
## Service Module Specification
This section explains how to author a clan service module. As decided in [01-clan-service-modules](https://git.clan.lol/clan/clan-core/src/branch/main/decisions/01-ClanModules.md)
!!! Warning
The described modules are fundamentally different to the existing [clanModules](../clanmodules/index.md)
Most of the clanModules will be migrated into the described format. We actively seek for contributions here.
### Minimal module
!!! Tip
Unlike previously modules can now be inlined. There is no file-system structure needed anymore.
First of all we need to hook our module into the `inventory.modules` attribute. Make sure to choose a unique name so the module doesn't have a name collision with any of the core modules.
While not required we recommend to prefix your module attribute name.
If you export the module from your flake, other people will be able to import it and use it within their clan.
i.e. `@hsjobeki/customNetworking`
```nix title=flake.nix
# ...
outputs = inputs: flake-parts.lib.mkFlake { inherit inputs; } ({
imports = [ inputs.clan-core.flakeModules.default ];
# ...
clan = {
inventory = {
# We could also inline the complete module spec here
# For example
# {...}: { _class = "clan.service"; ... };
modules."@hsjobeki/customNetworking" = import ./service-modules/networking.nix;
};
# If needed: Exporting the module for other people
modules."@hsjobeki/customNetworking" = import ./service-modules/networking.nix;
};
})
```
The imported module file must fulfill at least the following requirements:
- Be an actual module. That means: Be either an attribute set; or a function that returns an attribute set.
- Required `_class = "clan.service"
- Required `manifest.name = "<name of the provided service>"`
```nix title="/service-modules/networking.nix"
{
_class = "clan.service";
manifest.name = "zerotier-networking";
# ...
}
```
### Adding functionality to the module
While the very minimal module is valid in itself it has no way of adding any machines to it, because it doesn't specify any roles.
The next logical step is to think about the interactions between the machines and define *roles* for them.
Here is a short guide with some conventions:
- [ ] If they all have the same relation to each other `peer` is commonly used. `peers` can often talk to each other directly.
- [ ] Often machines don't necessarily have direct relation to each other and there is one elevated machine in the middle classically know as `client-server`. `clients` are less likely to talk directly to each other than `peers`
- [ ] If your machines don't have any relation and/or interactions to each other you should reconsider if the desired functionality is really a multi-host service.
```nix title="/service-modules/networking.nix"
{
_class = "clan.service";
manifest.name = "zerotier-networking";
# Define what roles exist
roles.peer = {};
roles.controller = {};
# ...
}
```
Next we need to define the settings and the behavior of these distinct roles.
```nix title="/service-modules/networking.nix"
{
_class = "clan.service";
manifest.name = "zerotier-networking";
# Define what roles exist
roles.peer = {
interface = {
# These options can be set via 'roles.client.settings'
options.ipRanges = mkOption { type = listOf str; };
};
# Maps over all instances and produces one result per instance.
perInstance = { instanceName, settings, machine, roles, ... }: {
# Analog to 'perSystem' of flake-parts.
# For every instance of this service we will add a nixosModule to a client-machine
nixosModule = { config, ... }: {
# Interaction examples what you could do here:
# - Get some settings of this machine
# settings.ipRanges
#
# - Get all controller names:
# allControllerNames = lib.attrNames roles.controller.machines
#
# - Get all roles of the machine:
# machine.roles
#
# - Get the settings that where applied to a specific controller machine:
# roles.controller.machines.jon.settings
#
# Add one systemd service for every instance
systemd.services.zerotier-client-${instanceName} = {
# ... depend on the '.config' and 'perInstance arguments'
};
};
}
};
roles.controller = {
interface = {
# These options can be set via 'roles.server.settings'
options.dynamicIp.enable = mkOption { type = bool; };
};
perInstance = { ... }: {};
};
# Maps over all machines and produces one result per machine.
perMachine = { instances, machine, ... }: {
# Analog to 'perSystem' of flake-parts.
# For every machine of this service we will add exactly one nixosModule to a machine
nixosModule = { config, ... }: {
# Interaction examples what you could do here:
# - Get the name of this machine
# machine.name
#
# - Get all roles of this machine across all instances:
# machine.roles
#
# - Get the settings of a specific instance of a specific machine
# instances.foo.roles.peer.machines.jon.settings
#
# Globally enable something
networking.enable = true;
};
};
# ...
}
```
## Using values from a NixOS machine inside the module
!!! Example "Experimental Status"
This feature is experimental and should be used with care.
Sometimes a settings value depends on something within a machines `config`.
Since the `interface` is defined completely machine-agnostic this means values from a machine cannot be set in the traditional way.
The following example shows how to create a local instance of machine specific settings.
```nix title="someservice.nix"
{
# Maps over all instances and produces one result per instance.
perInstance = { instanceName, extendSettings, machine, roles, ... }: {
nixosModule = { config, ... }:
let
# Create new settings with
# 'ipRanges' defaulting to 'config.network.ip.range' from this machine
# This only works if there is no 'default' already.
localSettings = extendSettings {
ipRanges = lib.mkDefault config.network.ip.range;
};
in
{
# ...
};
};
}
```
!!! Danger
`localSettings` are a local attribute. Other machines cannot access it.
If calling extendSettings is done that doesn't change the original `settings` this means if a different machine tries to access i.e `roles.client.settings` it would *NOT* contain your changes.
Exposing the changed settings to other machines would come with a huge performance penalty, thats why we don't want to offer it.

2
lib/inventory/build-inventory/interface.nix
View File

@@ -120,7 +120,7 @@ in
- The module MUST have at least `features = [ "inventory" ]` in the frontmatter section. - The module MUST have at least `features = [ "inventory" ]` in the frontmatter section.
- The module MUST have a subfolder `roles` with at least one `{roleName}.nix` file. - The module MUST have a subfolder `roles` with at least one `{roleName}.nix` file.
For further information see: [Module Authoring Guide](../../clanmodules/index.md). For further information see: [Module Authoring Guide](../../authoring/clanServices/index.md).
???+ example ???+ example
```nix ```nix