Merge pull request 'Docs: move {contributing, disk, mesh, backups} into guides' (#3692) from hsjobeki/clan-core:docs-structure into main

Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/3692

README.md

@@ -24,7 +24,7 @@ If you're new to Clan and eager to dive in, start with our quickstart guide and
 
 In the Clan ecosystem, security is paramount. Learn how to handle secrets effectively:
 
-- **Secrets Management**: Securely manage secrets by consulting [secrets](https://docs.clan.lol/getting-started/secrets/)<!-- [secrets.md](docs/site/getting-started/secrets.md) -->.
+- **Secrets Management**: Securely manage secrets by consulting [secrets](https://docs.clan.lol/guides/getting-started/secrets/)<!-- [secrets.md](docs/site/guides/getting-started/secrets.md) -->.
 
 ### Contributing to Clan
 

docs/mkdocs.yml

@@ -46,21 +46,20 @@ exclude_docs: |
 
 nav:
   - Home: index.md
-  - Getting Started:
-      - Setup Clan: getting-started/index.md
-      - Create Installer: getting-started/installer.md
-      - Add Machines: getting-started/configure.md
-      - Secrets & Facts: getting-started/secrets.md
-      - Deploy Machine: getting-started/deploy.md
-      - Continuous Integration: getting-started/check.md
   - Guides:
+      - Getting Started:
+          - Setup Clan: guides/getting-started/index.md
+          - Create Installer: guides/getting-started/installer.md
+          - Add Machines: guides/getting-started/configure.md
+          - Secrets & Facts: guides/getting-started/secrets.md
+          - Deploy Machine: guides/getting-started/deploy.md
+          - Continuous Integration: guides/getting-started/check.md
       - clanServices: guides/clanServices.md
-      - Disk Encryption: getting-started/disk-encryption.md
-      - Mesh VPN: getting-started/mesh-vpn.md
-      - Backup & Restore: getting-started/backups.md
+      - Disk Encryption: guides/disk-encryption.md
+      - Mesh VPN: guides/mesh-vpn.md
+      - Backup & Restore: guides/backups.md
       - Vars Backend: manual/vars-backend.md
       - Facts Backend: manual/secrets.md
-      - Facts Vars Migration: manual/migration-facts-vars.md
       - Autoincludes: manual/adding-machines.md
       - Inventory:
           - Inventory: manual/inventory.md
@@ -71,20 +70,15 @@ nav:
           - Disk Template: guides/authoring/templates/disk/disko-templates.md
           - clanModule: guides/authoring/clanModules/index.md
       - Contributing:
-          - Contribute: contributing/contribute.md
-          - Debugging: contributing/debugging.md
-          - Testing: contributing/testing.md
-      - Repo Layout: manual/repo-layout.md
-      - Migrate existing Flakes: manual/migration-guide.md
-      - Migrate inventory Services: guides/migrate-inventory-services.md
+          - Contribute: guides/contributing/CONTRIBUTING.md
+          - Debugging: guides/contributing/debugging.md
+          - Testing: guides/contributing/testing.md
+      - Migrations:
+          - Migrate existing Flakes: guides/migrations/migration-guide.md
+          - Migrate inventory Services: guides/migrations/migrate-inventory-services.md
+          - Facts Vars Migration: guides/migrations/migration-facts-vars.md
   - Reference:
       - Overview: reference/index.md
-      - Architecture Decisions:
-          - Architecture Decisions: decisions/README.md
-          - 01-clanModules: decisions/01-ClanModules.md
-          - 02-clan-api: decisions/02-clan-api.md
-          - 03-adr-numbering-process: decisions/03-adr-numbering-process.md
-          - Template: decisions/_template.md
       - Clan Services:
           - reference/clanServices/index.md
           - reference/clanServices/admin.md
@@ -165,6 +159,12 @@ nav:
       - Nix API:
           - buildClan: reference/nix-api/buildclan.md
           - Inventory: reference/nix-api/inventory.md
+      - Decisions:
+          - Architecture Decisions: decisions/README.md
+          - 01-clanModules: decisions/01-ClanModules.md
+          - 02-clan-api: decisions/02-clan-api.md
+          - 03-adr-numbering-process: decisions/03-adr-numbering-process.md
+          - Template: decisions/_template.md
 
 docs_dir: site
 site_dir: out

docs/nix/render_options/__init__.py

@@ -457,7 +457,7 @@ Each `clanService`:
 !!! Note
     `clanServices` are part of Clan's next-generation service model and are intended to replace `clanModules`.
 
-    See [Migration Guide](../../guides/migrate-inventory-services.md) for help on migrating.
+    See [Migration Guide](../../guides/migrations/migrate-inventory-services.md) for help on migrating.
 
 Learn how to use `clanServices` in practice in the [Using clanServices guide](../../guides/clanServices.md).
 """

docs/site/contributing/contribute.md

@@ -1 +0,0 @@
-../../CONTRIBUTING.md

docs/site/drafts/deployment-framework.md

@@ -1,63 +0,0 @@
----
-title: "Git Based Machine Deployment with Clan-Core"
-description: ""
-authors:
-  - Qubasa
-date: 2024-05-25
----
-## Revolutionizing Server Management
-
-In the world of server management, countless tools claim to offer seamless deployment of multiple machines. Yet, many fall short, leaving server admins and self-hosting enthusiasts grappling with complexity. Enter the Clan-Core Framework—a groundbreaking all in one solution designed to transform decentralized self-hosting into an effortless and scalable endeavor.
-
-### The Power of Clan-Core
-
-Imagine having the power to manage your servers with unparalleled ease, scaling your IT infrastructure like never before. Clan-Core empowers you to do just that. At its core, Clan-Core leverages a single Git repository to define everything about your machines. This central repository utilizes Nix or JSON files to specify configurations, including disk formatting, ensuring a streamlined and unified approach.
-
-### Simplified Deployment Process
-
-With Clan-Core, the cumbersome task of bootstrapping a specific ISO is a thing of the past. All you need is SSH access to your Linux server. Clan-Core allows you to overwrite any existing Linux distribution live over SSH, eliminating time-consuming setup processes. This capability means you can deploy updates or new configurations swiftly and efficiently, maximizing uptime and minimizing hassle.
-
-### Secure and Efficient Secret Management
-
-Security is paramount in server management, and Clan-Core takes it seriously. Passwords and other sensitive information are encrypted within the Git repository, automatically decrypted during deployment. This not only ensures the safety of your secrets but also simplifies their management. Clan-Core supports sharing secrets with other admins, fostering collaboration and maintaining reproducibillity and security without sacrificing convenience.
-
-### Services as Apps
-
-Setting up a service can be quite difficult. Many server adjustments need to be made, from setting up a database to adjusting webserver configurations and generating the correct private keys. However, Clan-Core aims to make setting up a service as easy as installing an application. Through Clan-Core's Module system, everything down to secrets can be automatically set up. This transforms the often daunting task of service setup into a smooth, automated process, making it accessible to all.
-
-### Decentralized Mesh VPN
-
-Building on these features is a self-configuring decentralized mesh VPN that interconnects all your machines into a private darknet. This ensures that sensitive services, which might have too much attack surface to be hosted on the public internet, can still be made available privately without the need to worry about potential system compromise. By creating a secure, private network, Clan-Core offers an additional layer of protection for your most critical services.
-
-### Decentralized Domain Name System
-
-Current DNS implementations are distributed but not truly decentralized. For Clan-Core, we implemented our own truly decentralized DNS module. This module uses simple flooding and caching algorithms to discover available domains inside the darknet. This approach ensures that your internal domain name system is robust, reliable, and independent of external control, enhancing the resilience and security of your infrastructure.
-
-
-### A New Era of Decentralized Self-Hosting
-
-Clan-Core is more than just a tool; it's a paradigm shift in server management. By consolidating machine definitions, secrets and network configuration, into a single, secure repository, it transforms how you manage and scale your infrastructure. Whether you're a seasoned server admin or a self-hosting enthusiast, Clan-Core offers a powerful, user-friendly solution to take your capabilities to the next level.
-
-
-### Key Features of Clan-Core:
-
-- **Unified Git Repository**: All machine configurations and secrets stored in a single repository.
-- **Live Overwrites**: Deploy configurations over existing Linux distributions via SSH.
-- **Automated Service Setup**: Easily set up services with Clan-Core's Module system.
-- **Decentralized Mesh VPN**: Securely interconnect all machines into a private darknet.
-- **Decentralized DNS**: Robust, independent DNS using flooding and caching algorithms.
-- **Automated Secret Management**: Encrypted secrets that are automatically decrypted during deployment.
-- **Collaboration Support**: Share secrets securely with other admins.
-
-
-## Clan-Cores Future
-
-Our vision for Clan-Core extends far beyond being just another deployment tool. Clan-Core is a framework we've developed to achieve something much greater. We want to put the "personal" back into "personal computing." Our goal is for everyday users to fully customize their phones or laptops and create truly private spaces for friends and family.
-
-Our first major step is to develop a Graphical User Interface (GUI) that makes configuring all this possible. Initial tests have shown that AI can be leveraged as an alternative to traditional GUIs. This paves the way for a future where people can simply talk to their computers, and they will configure themselves according to the users' wishes. 
-
-By adopting Clan, you're not just embracing a tool—you're joining a movement towards a more efficient, secure, and scalable approach to server management. Join us and revolutionize your IT infrastructure today.
-
-
-
-

docs/site/guides/clanServices.md

@@ -125,5 +125,5 @@ You might expose your service module from your flake — this makes it easy for
 ## What’s Next?
 
 * [Author your own clanService →](../guides/authoring/clanServices/index.md)
-* [Migrate from clanModules →](../guides/migrate-inventory-services.md)
+* [Migrate from clanModules →](../guides/migrations/migrate-inventory-services.md)
 <!-- TODO: * [Understand the architecture →](../explanation/clan-architecture.md) -->

docs/site/guides/contributing/CONTRIBUTING.md

@@ -0,0 +1 @@
+../../../CONTRIBUTING.md

docs/site/guides/getting-started/configure.md

@@ -14,7 +14,7 @@ Clan currently offers the following methods to configure machines:
 
     - machines/`machine_name`/configuration.nix (`autoincluded` if it exists)
 
-    See the complete [list](../manual/adding-machines.md#automatic-registration) of auto-loaded files.
+    See the complete [list](../../manual/adding-machines.md#automatic-registration) of auto-loaded files.
 
 ???+ Note "Used by CLI & UI"
 
@@ -40,7 +40,7 @@ In the `flake.nix` file:
 
 === "**template using flake-parts**"
 
-    !!! info "See [Clan with flake-parts](../manual/flake-parts.md) for help migrating to flake-parts."
+    !!! info "See [Clan with flake-parts](../../manual/flake-parts.md) for help migrating to flake-parts."
 
     ```nix title="flake.nix" hl_lines="3"
     clan = {
@@ -106,7 +106,7 @@ Adding or configuring a new machine requires two simple steps:
       clan.core.networking.targetHost = "root@__IP__";
 
 
-      # Replace this __CHANGE_ME__ with the result of the lsblk command from step 1. 
+      # Replace this __CHANGE_ME__ with the result of the lsblk command from step 1.
       disko.devices.disk.main.device = "/dev/disk/by-id/__CHANGE_ME__";
 
       # IMPORTANT! Add your SSH key here

docs/site/guides/getting-started/index.md

@@ -45,7 +45,7 @@ Add the Clan CLI into your development workflow:
 nix shell git+https://git.clan.lol/clan/clan-core#clan-cli --refresh
 ```
 
-You can find reference documentation for the `clan` cli program [here](../reference/cli/index.md).
+You can find reference documentation for the `clan` cli program [here](../../reference/cli/index.md).
 
 Alternatively you can check out the help pages directly:
 ```terminalSession
@@ -54,7 +54,7 @@ clan --help
 
 ### Step 2: Initialize Your Project
 
-If you want to migrate an existing project, follow this [guide](https://docs.clan.lol/manual/migration-guide/).
+If you want to migrate an existing project, follow this [guide](../migrations/migration-guide.md).
 
 Set the foundation of your Clan project by initializing it as follows:
 
@@ -104,8 +104,8 @@ This should yield the following:
     That way you will have the tool available in the shell environment.
     We also recommend setting up [direnv](https://direnv.net/) for your shell, for a more convenient
     experience.
-    
-    
+
+
 
 ```bash
 clan machines list

docs/site/guides/getting-started/installer.md

@@ -151,7 +151,7 @@ sudo umount /dev/sdb1
 
 
 ###  Step 3: Boot From USB Stick
-- To use, boot from the Clan USB drive with **secure boot turned off**. For step by step instructions go to [Disabling Secure Boot](../manual/secure-boot.md)
+- To use, boot from the Clan USB drive with **secure boot turned off**. For step by step instructions go to [Disabling Secure Boot](../../manual/secure-boot.md)
 
 
 ## (Optional) Connect to Wifi Manually

docs/site/guides/migrations/migrate-inventory-services.md

@@ -1,7 +1,7 @@
 # Migrating from using `clanModules` to `clanServices`
 
 **Audience**: This is a guide for **people using `clanModules`**.
-If you are a **module author** and need to migrate your modules please consult our **new** [clanServices authoring guide](../guides/authoring/clanServices/index.md)
+If you are a **module author** and need to migrate your modules please consult our **new** [clanServices authoring guide](../authoring/clanServices/index.md)
 
 ## What's Changing?
 
@@ -78,7 +78,7 @@ instances = {
 
 ### 1. Move `services` entries to `instances`
 
-Check if a service that you use has been migrated [In our reference](../reference/clanServices/index.md)
+Check if a service that you use has been migrated [In our reference](../../reference/clanServices/index.md)
 
 In your inventory, move it from:
 
@@ -142,6 +142,6 @@ roles.default.machines."test-inventory-machine".settings = {
 
 ## Further reference
 
-* [Authoring a 'clan.service' module](../guides/authoring/clanServices/index.md)
-* [ClanServices](../guides/clanServices.md)
-* [Inventory Reference](../reference/nix-api/inventory.md)
+* [Authoring a 'clan.service' module](../authoring/clanServices/index.md)
+* [ClanServices](../clanServices.md)
+* [Inventory Reference](../../reference/nix-api/inventory.md)

docs/site/guides/migrations/migration-facts-vars.md

@@ -2,10 +2,10 @@
 
 For a high level overview about `vars` see our [blog post](https://clan.lol/blog/vars/).
 
-This guide will help you migrate your modules that still use our [`facts`](../manual/secrets.md) backend
-to the [`vars`](../manual/vars-backend.md) backend.
+This guide will help you migrate your modules that still use our [`facts`](../../manual/secrets.md) backend
+to the [`vars`](../../manual/vars-backend.md) backend.
 
-The `vars` [module](../reference/clan-core/vars.md) and the clan [command](../reference/cli/vars.md) work in tandem, they should ideally be kept in sync.
+The `vars` [module](../../reference/clan-core/vars.md) and the clan [command](../../reference/cli/vars.md) work in tandem, they should ideally be kept in sync.
 
 ## Keep Existing Values
 

docs/site/guides/migrations/migration-guide.md

@@ -1,6 +1,6 @@
 # Migrate existing NixOS configurations
 
-This guide will help you migrate your existing Nix configurations into Clan. 
+This guide will help you migrate your existing Nix configurations into Clan.
 
 !!! Warning
     Migrating instead of starting new can be trickier and might lead to bugs or
@@ -77,9 +77,9 @@ For the provide flake example, your flake should now look like this:
 ```nix
 {
   inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
-  
+
   inputs.clan-core = {
-    url = "git+https://git.clan.lol/clan/clan-core";    
+    url = "git+https://git.clan.lol/clan/clan-core";
     inputs.nixpkgs.follows = "nixpkgs";
   };
 
@@ -104,9 +104,9 @@ For the provide flake example, your flake should now look like this:
   in
   {
       nixosConfigurations = clan.nixosConfigurations;
-      
+
       inherit (clan) clanInternals;
-      
+
       clan = {
         inherit (clan) templates;
       };

docs/site/index.md

@@ -6,32 +6,7 @@ hide:
 
 # :material-home: Welcome to **Clan**'s  documentation
 
-[Getting Started](./getting-started/index.md){ .md-button }
-
-## Tutorials
-
-**Learning-oriented adventures with a hands-on experience.**
-
-<div class="grid cards" markdown>
-
--   :material-clock-fast:{ .lg .middle } __Set up in 15 minutes__
-
-    ---
-
-    Create your own clan and get everything
-    running in minutes
-
-    [:octicons-arrow-right-24: Getting started](./getting-started/index.md)
-
--   :fontawesome-solid-user-group:{ .lg .middle } __Authoring Modules__
-
-    ---
-
-    Create ressources that can be reused by the community.
-
-    [:octicons-arrow-right-24: Authoring guides](./guides/authoring/clanModules/index.md)
-
-</div>
+[Getting Started](./guides/getting-started/index.md){ .md-button }
 
 ## :material-book: Guides
 
@@ -63,7 +38,7 @@ hide:
 
     Use clan with [https://flake.parts/]()
 
--   [Contribute](./contributing/contribute.md)
+-   [Contribute](./guides/contributing/CONTRIBUTING.md)
 
     ---
 
@@ -77,23 +52,17 @@ hide:
 
 <div class="grid cards" markdown>
 
--   [Reference Overview](./reference/index.md)
-
-    ---
-
-    Learn how to interface with Clan programmatically
-
 -   [Cli Reference](./reference/cli/index.md)
 
     ---
 
     The `clan` cli command
 
--   [Modules](./reference/clanModules/index.md)
+-   [Service Modules](./reference/clanServices/index.md)
 
     ---
 
-    An overview of available clan modules
+    An overview of available service modules
 
 -   [Core](./reference/clan-core/index.md)
 
@@ -103,5 +72,13 @@ hide:
     This is imported when using clan and is the basis of the extra functionality
     that can be provided.
 
+-   [(Legacy) Modules](./reference/clanModules/index.md)
+
+    ---
+
+    An overview of available clanModules
+
+    !!! Example "Those will get deprecated soon"
+
 
 </div>

docs/site/manual/repo-layout.md

@@ -1,20 +0,0 @@
-
-This guide will help you navigate the codebase and locate key files:
-
-```bash
-$ tree -L 1
-.
-├── checks          # Contains NixOS and VM tests
-├── clanModules     # Clan modules available for end-user import
-├── docs            # Source files for docs.clan.lol, generated with MkDocs
-├── flakeModules
-├── lib             # User-exposed Clan Nix functions like buildClan and inventory
-├── machines
-├── nixosModules    # Internal Clan Nix functions, e.g., clanCore
-├── pkgs            # Clan applications and packaged dependencies
-├── formatter.nix   # Configuration for nix-treefmt, manages `nix fmt`
-├── scripts
-├── sops
-├── templates       # Template files for creating a new Clan
-└── vars
-```

docs/site/manual/vars-backend.md

@@ -10,7 +10,7 @@ In this example, we will guide you through automating that interaction using cla
 For a more general explanation of what clan vars are and how it works, see the intro of the [Reference Documentation for vars](https://docs.clan.lol/reference/clan-core/vars/)
 
 This guide assumes
-- clan is set up already (see [Getting Started](../getting-started/index.md))
+- clan is set up already (see [Getting Started](../guides/getting-started/index.md))
 - a machine has been added to the clan (see [Adding Machines](./adding-machines.md))
 
 This section will walk you through the following steps: