Merge remote-tracking branch 'origin/main' into rework-installation

2024-07-30 11:52:36 +02:00
parent 45124bf324 e20c9d52fd
commit cc5546d79d
48 changed files with 1385 additions and 562 deletions
--- a/.envrc
+++ b/.envrc
@@ -4,6 +4,7 @@ if ! has nix_direnv_version || ! nix_direnv_version 3.0.4; then
 fi

 watch_file .direnv/selected-shell 
+watch_file formatter.nix

 if [ -e .direnv/selected-shell ]; then
  use flake ".#$(cat .direnv/selected-shell)"
--- a/docs/nix/flake-module.nix
+++ b/docs/nix/flake-module.nix
@@ -54,9 +54,11 @@
      module-docs =
        pkgs.runCommand "rendered"
          {
-            nativeBuildInputs = [
+            buildInputs = [
              pkgs.python3
              self'.packages.clan-cli
+              # TODO: see postFixup clan-cli/default.nix:L188
+              self'.packages.clan-cli.propagatedBuildInputs
            ];
          }
          ''
--- a/docs/site/blog/.authors.yml
+++ b/docs/site/blog/.authors.yml
@@ -24,3 +24,8 @@ authors:
    description: "Core Developer"
    avatar: "https://clan.lol/static/profiles/qubasa.png"
    url: "https://github.com/Qubasa"
+  BrianMcGee:
+    name: "Brian McGee"
+    description: "Contributor"
+    avatar: "https://avatars.githubusercontent.com/u/1173648?v=4"
+    url: "https://bmcgee.ie"
--- a/docs/site/blog/posts/nixos-facter.md
+++ b/docs/site/blog/posts/nixos-facter.md
@@ -0,0 +1,100 @@
+---
+title: "Introducing NixOS Facter"
+description: "Declarative Hardware Configuration in NixOS"
+authors:
+  - BrianMcGee
+date: 2024-07-19
+slug: nixos-facter
+---
+
+If you've ever installed [NixOS], you'll be familiar with a little Perl script called [nixos-generate-config]. Unsurprisingly, it generates a couple of NixOS modules based on available hardware, mounted filesystems, configured swap, etc.
+
+It's a critical component of the install process, aiming to ensure you have a good starting point for your NixOS system, with necessary or recommended kernel modules, file system mounts, networking config and much more.
+
+As solutions go, it's a solid one. It has helped many users take their first steps into this rabbit hole we call NixOS. However, it does suffer from one fundamental limitation.
+
+## Static Generation
+
+When a user generates a `hardware-configuration.nix` with `nixos-generate-config`, it makes choices based on the current state of the world as it sees it. By its very nature, then, it cannot account for changes in NixOS over time.
+
+A recommended configuration option today might be different two NixOS releases from now.
+
+To account for this, you could always run `nixos-generate-config` again. But that requires a working system, which may have broken due to the historical choices made last time, or worst-case, requiring you to fire up the installer again.
+
+## A Layer of Indirection
+
+What if, instead of generating some Nix code, we first describe the current hardware in an intermediate format? This hardware report would be _'pure'_, devoid of any reference to NixOS, and intended as a stable, longer-term representation of the system.
+
+From here, we can create a series of NixOS modules designed to examine the report's contents and make the same kinds of decisions that `nixos-generate-config` does. The critical difference is that as NixOS evolves, so can these modules, and with a full hardware report available we can make more interesting config choices about things such as GPUs and other devices.
+
+In a perfect world, we should not need to regenerate the underlying report as long as there are no hardware changes. We can take this one step further.
+
+Provided that certain sensitive information, such as serial numbers and MAC addresses, is filtered out, there is no reason why these hardware reports could not be shared after they are generated for things like EC2 instance types, specific laptop models, and so on, much like [NixOS Hardware] currently shares Nix configs.
+
+## Introducing NixOS Facter
+
+Still in its early stages, [NixOS Facter] is intended to do what I've described above.
+
+A user can generate a JSON-based hardware report using a (eventually static) Go program: `nixos-facter -o facter.json`. From there, they can include this report in their NixOS config and make use of our [NixOS modules](https://github.com/numtide/nixos-facter-modules) as follows:
+
+=== "**flake.nix**"
+
+    ```nix
+    {
+        inputs = {
+            nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
+            nixos-facter-modules.url = "github:numtide/nixos-facter-modules";
+        };
+
+        outputs = inputs @ {
+            nixpkgs,
+            ...
+        }: {
+            nixosConfigurations.basic = nixpkgs.lib.nixosSystem {
+                modules = [
+                    inputs.nixos-facter-modules.nixosModules.facter
+                    { config.facter.reportPath = ./facter.json; }
+                    # ...
+                ];
+            };
+        };
+    }
+    ```
+
+=== "**without flakes**"
+
+    ```nix
+    # configuration.nix
+    {
+        imports = [
+          "${(builtins.fetchTarball {
+            url = "https://github.com/numtide/nixos-facter-modules/";
+          })}/modules/nixos/facter.nix"
+        ];
+
+        config.facter.reportPath = ./facter.json;
+    }
+    ```
+
+That's it.
+
+> We assume that users will rely on [disko], so we have not implemented file system configuration yet (it's on the roadmap). 
+> In the meantime, if you don't use disko you have to specify that part of the configuration yourself or take it from `nixos-generate-config`.
+
+
+## Early Days
+
+Please be aware that [NixOS Facter] is still in early development and is still subject to significant changes especially the output json format as we flesh things out. Our initial goal is to reach feature parity with [nixos-generate-config].
+
+From there, we want to continue building our NixOS modules, opening things up to the community, and beginning to capture shared hardware configurations for providers such as Hetzner, etc.
+
+Over the coming weeks, we will also build up documentation and examples to make it easier to play with. For now, please be patient.
+
+> Side note: if you are wondering why the repo is in the [Numtide] org, we started partnering with Clan! Both companies are looking to make self-hosting easier and we're excited to be working together on this. Expect more tools and features to come!
+
+[NixOS Facter]: https://github.com/numtide/nixos-facter
+[NixOS Hardware]: https://github.com/NixOS/nixos-hardware
+[NixOS]: https://nixos.org "Declarative builds and deployments"
+[Numtide]: https://numtide.com
+[disko]: https://github.com/nix-community/disko
+[nixos-generate-config]: https://github.com/NixOS/nixpkgs/blob/dac9cdf8c930c0af98a63cbfe8005546ba0125fb/nixos/modules/installer/tools/nixos-generate-config.pl
--- a/formatter.nix
+++ b/formatter.nix
@@ -30,8 +30,8 @@
            {
              "pkgs/clan-vm-manager" = {
                extraPythonPackages =
-                  # clan-app currently only exists on linux
-                  self'.packages.clan-vm-manager.testDependencies ++ self'.packages.clan-cli.testDependencies;
+                  #   # clan-app currently only exists on linux
+                  self'.packages.clan-vm-manager.testDependencies;
                modules = [ "clan_vm_manager" ];
              };
            }
--- a/nixosModules/clanCore/vars/default.nix
+++ b/nixosModules/clanCore/vars/default.nix
@@ -18,7 +18,7 @@ in
    ./public/in_repo.nix
    # ./public/vm.nix
    ./secret/password-store.nix
-    ./secret/sops.nix
+    ./secret/sops
    # ./secret/vm.nix
  ];
  options.clan.core.vars = lib.mkOption {
--- a/nixosModules/clanCore/vars/interface.nix
+++ b/nixosModules/clanCore/vars/interface.nix
@@ -72,7 +72,7 @@ in
                        name of the generator
                      '';
                      readOnly = true;
-                      default = generator.name;
+                      default = generator.config._module.args.name;
                    };
                    secret = {
                      description = ''
@@ -87,7 +87,6 @@ in
                        This will be set automatically
                      '';
                      type = str;
-                      readOnly = true;
                    };
                    value = {
                      description = ''
@@ -109,32 +108,35 @@ in
                For example, a prompt named 'prompt1' will be available via $prompts/prompt1
              '';
              default = { };
-              type = attrsOf (submodule {
-                options = options {
-                  description = {
-                    description = ''
-                      The description of the prompted value
-                    '';
-                    type = str;
-                    example = "SSH private key";
+              type = attrsOf (
+                submodule (prompt: {
+                  options = options {
+                    description = {
+                      description = ''
+                        The description of the prompted value
+                      '';
+                      type = str;
+                      example = "SSH private key";
+                      default = prompt.config._module.args.name;
+                    };
+                    type = {
+                      description = ''
+                        The input type of the prompt.
+                        The following types are available:
+                          - hidden: A hidden text (e.g. password)
+                          - line: A single line of text
+                          - multiline: A multiline text
+                      '';
+                      type = enum [
+                        "hidden"
+                        "line"
+                        "multiline"
+                      ];
+                      default = "line";
+                    };
                  };
-                  type = {
-                    description = ''
-                      The input type of the prompt.
-                      The following types are available:
-                        - hidden: A hidden text (e.g. password)
-                        - line: A single line of text
-                        - multiline: A multiline text
-                    '';
-                    type = enum [
-                      "hidden"
-                      "line"
-                      "multiline"
-                    ];
-                    default = "line";
-                  };
-                };
-              });
+                })
+              );
            };
            runtimeInputs = {
              description = ''
--- a/nixosModules/clanCore/vars/public/in_repo.nix
+++ b/nixosModules/clanCore/vars/public/in_repo.nix
@@ -5,8 +5,9 @@
      {
        publicModule = "clan_cli.vars.public_modules.in_repo";
        fileModule = file: {
-          path =
-            config.clan.core.clanDir + "/machines/${config.clan.core.machineName}/vars/${file.config.name}";
+          path = lib.mkIf (file.config.secret == false) (
+            config.clan.core.clanDir + "/machines/${config.clan.core.machineName}/vars/${file.config.name}"
+          );
        };
      };
 }
--- a/nixosModules/clanCore/vars/secret/password-store.nix
+++ b/nixosModules/clanCore/vars/secret/password-store.nix
@@ -4,7 +4,7 @@
    lib.mkIf (config.clan.core.vars.settings.secretStore == "password-store")
      {
        fileModule = file: {
-          path = lib.mkIf file.secret "${config.clan.core.password-store.targetDirectory}/${config.clan.core.machineName}-${file.config.generatorName}-${file.config.name}";
+          path = lib.mkIf file.config.secret "${config.clan.core.password-store.targetDirectory}/${config.clan.core.machineName}-${file.config.generatorName}-${file.config.name}";
        };
        secretUploadDirectory = lib.mkDefault "/etc/secrets";
        secretModule = "clan_cli.vars.secret_modules.password_store";
--- a/nixosModules/clanCore/vars/secret/sops.nix
+++ b/nixosModules/clanCore/vars/secret/sops.nix
@@ -1,61 +0,0 @@
-{
-  config,
-  lib,
-  pkgs,
-  ...
-}:
-let
-  secretsDir = config.clan.core.clanDir + "/sops/secrets";
-  groupsDir = config.clan.core.clanDir + "/sops/groups";
-
-  # My symlink is in the nixos module detected as a directory also it works in the repl. Is this because of pure evaluation?
-  containsSymlink =
-    path:
-    builtins.pathExists path
-    && (builtins.readFileType path == "directory" || builtins.readFileType path == "symlink");
-
-  containsMachine =
-    parent: name: type:
-    type == "directory" && containsSymlink "${parent}/${name}/machines/${config.clan.core.machineName}";
-
-  containsMachineOrGroups =
-    name: type:
-    (containsMachine secretsDir name type)
-    || lib.any (
-      group: type == "directory" && containsSymlink "${secretsDir}/${name}/groups/${group}"
-    ) groups;
-
-  filterDir =
-    filter: dir:
-    lib.optionalAttrs (builtins.pathExists dir) (lib.filterAttrs filter (builtins.readDir dir));
-
-  groups = builtins.attrNames (filterDir (containsMachine groupsDir) groupsDir);
-  secrets = filterDir containsMachineOrGroups secretsDir;
-in
-{
-  config.clan.core.vars.settings = lib.mkIf (config.clan.core.vars.settings.secretStore == "sops") {
-    # Before we generate a secret we cannot know the path yet, so we need to set it to an empty string
-    fileModule = file: {
-      path =
-        lib.mkIf file.secret
-          config.sops.secrets.${"vars-${config.clan.core.machineName}-${file.config.generatorName}-${file.config.name}"}.path
-            or "/no-such-path";
-    };
-    secretModule = "clan_cli.vars.secret_modules.sops";
-    secretUploadDirectory = lib.mkDefault "/var/lib/sops-nix";
-  };
-
-  config.sops = lib.mkIf (config.clan.core.vars.settings.secretStore == "sops") {
-    secrets = builtins.mapAttrs (name: _: {
-      sopsFile = config.clan.core.clanDir + "/sops/secrets/${name}/secret";
-      format = "binary";
-    }) secrets;
-    # To get proper error messages about missing secrets we need a dummy secret file that is always present
-    defaultSopsFile = lib.mkIf config.sops.validateSopsFiles (
-      lib.mkDefault (builtins.toString (pkgs.writeText "dummy.yaml" ""))
-    );
-    age.keyFile = lib.mkIf (builtins.pathExists (
-      config.clan.core.clanDir + "/sops/secrets/${config.clan.core.machineName}-age.key/secret"
-    )) (lib.mkDefault "/var/lib/sops-nix/key.txt");
-  };
-}
--- a/nixosModules/clanCore/vars/secret/sops/default.nix
+++ b/nixosModules/clanCore/vars/secret/sops/default.nix
@@ -0,0 +1,49 @@
+{
+  config,
+  lib,
+  pkgs,
+  ...
+}:
+let
+
+  inherit (lib) flip;
+
+  inherit (import ./funcs.nix { inherit lib; }) listVars;
+
+  varsDir = config.clan.core.clanDir + "/sops/vars";
+
+  vars = listVars varsDir;
+
+in
+{
+  config.clan.core.vars.settings = lib.mkIf (config.clan.core.vars.settings.secretStore == "sops") {
+    # Before we generate a secret we cannot know the path yet, so we need to set it to an empty string
+    fileModule = file: {
+      path = lib.mkIf file.config.secret (
+        config.sops.secrets.${"${config.clan.core.machineName}/${file.config.generatorName}/${file.config.name}"}.path
+          or "/no-such-path"
+      );
+    };
+    secretModule = "clan_cli.vars.secret_modules.sops";
+    secretUploadDirectory = lib.mkDefault "/var/lib/sops-nix";
+  };
+
+  config.sops = lib.mkIf (config.clan.core.vars.settings.secretStore == "sops") {
+    secrets = lib.listToAttrs (
+      flip map vars (secret: {
+        name = secret.id;
+        value = {
+          sopsFile = config.clan.core.clanDir + "/sops/vars/${secret.id}/secret";
+          format = "binary";
+        };
+      })
+    );
+    # To get proper error messages about missing secrets we need a dummy secret file that is always present
+    defaultSopsFile = lib.mkIf config.sops.validateSopsFiles (
+      lib.mkDefault (builtins.toString (pkgs.writeText "dummy.yaml" ""))
+    );
+    age.keyFile = lib.mkIf (builtins.pathExists (
+      config.clan.core.clanDir + "/sops/secrets/${config.clan.core.machineName}-age.key/secret"
+    )) (lib.mkDefault "/var/lib/sops-nix/key.txt");
+  };
+}
--- a/nixosModules/clanCore/vars/secret/sops/eval-tests/default.nix
+++ b/nixosModules/clanCore/vars/secret/sops/eval-tests/default.nix
@@ -0,0 +1,43 @@
+{
+  lib ? import <nixpkgs/lib>,
+  pkgs ? import <nixpkgs> { },
+}:
+let
+  inherit (import ../funcs.nix { inherit lib; }) readDirNames listVars;
+
+  noVars = pkgs.runCommand "empty-dir" { } ''
+    mkdir $out
+  '';
+
+  emtpyVars = pkgs.runCommand "empty-dir" { } ''
+    mkdir -p $out/vars
+  '';
+
+in
+{
+  test_readDirNames = {
+    expr = readDirNames ./populated/vars;
+    expected = [ "my_machine" ];
+  };
+
+  test_listSecrets = {
+    expr = listVars ./populated/vars;
+    expected = [
+      {
+        machine = "my_machine";
+        generator = "my_generator";
+        name = "my_secret";
+      }
+    ];
+  };
+
+  test_listSecrets_no_vars = {
+    expr = listVars noVars;
+    expected = [ ];
+  };
+
+  test_listSecrets_empty_vars = {
+    expr = listVars emtpyVars;
+    expected = [ ];
+  };
+}
--- a/nixosModules/clanCore/vars/secret/sops/eval-tests/populated/vars/my_machine/my_generator/my_secret
+++ b/nixosModules/clanCore/vars/secret/sops/eval-tests/populated/vars/my_machine/my_generator/my_secret
--- a/nixosModules/clanCore/vars/secret/sops/funcs.nix
+++ b/nixosModules/clanCore/vars/secret/sops/funcs.nix
@@ -0,0 +1,29 @@
+{
+  lib ? import <nixpkgs/lib>,
+  ...
+}:
+let
+  inherit (builtins) readDir;
+
+  inherit (lib) concatMap flip;
+in
+rec {
+  readDirNames =
+    dir:
+    if !(builtins.pathExists dir) then [ ] else lib.mapAttrsToList (name: _type: name) (readDir dir);
+
+  listVars =
+    varsDir:
+    flip concatMap (readDirNames varsDir) (
+      machine_name:
+      flip concatMap (readDirNames (varsDir + "/${machine_name}")) (
+        generator_name:
+        flip map (readDirNames (varsDir + "/${machine_name}/${generator_name}")) (secret_name: {
+          machine = machine_name;
+          generator = generator_name;
+          name = secret_name;
+          id = "${machine_name}/${generator_name}/${secret_name}";
+        })
+      )
+    );
+}
--- a/nixosModules/clanCore/vars/settings-opts.nix
+++ b/nixosModules/clanCore/vars/settings-opts.nix
@@ -30,6 +30,7 @@
    '';
  };

+  # TODO: see if this is the right approach. Maybe revert to secretPathFunction
  fileModule = lib.mkOption {
    type = lib.types.deferredModule;
    internal = true;
--- a/pkgs/clan-app/clan_app/components/serializer.py
+++ b/pkgs/clan-app/clan_app/components/serializer.py
@@ -1,92 +0,0 @@
-import dataclasses
-import logging
-from dataclasses import fields, is_dataclass
-from pathlib import Path
-from types import UnionType
-from typing import Any, get_args
-
-import gi
-
-gi.require_version("WebKit", "6.0")
-
-log = logging.getLogger(__name__)
-
-
-def sanitize_string(s: str) -> str:
-    return s.replace("\\", "\\\\").replace('"', '\\"').replace("\n", "\\n")
-
-
-def dataclass_to_dict(obj: Any) -> Any:
-    """
-    Utility function to convert dataclasses to dictionaries
-    It converts all nested dataclasses, lists, tuples, and dictionaries to dictionaries
-
-    It does NOT convert member functions.
-    """
-    if dataclasses.is_dataclass(obj):
-        return {
-            sanitize_string(k): dataclass_to_dict(v)
-            for k, v in dataclasses.asdict(obj).items()
-        }
-    elif isinstance(obj, list | tuple):
-        return [dataclass_to_dict(item) for item in obj]
-    elif isinstance(obj, dict):
-        return {sanitize_string(k): dataclass_to_dict(v) for k, v in obj.items()}
-    elif isinstance(obj, Path):
-        return str(obj)
-    elif isinstance(obj, str):
-        return sanitize_string(obj)
-    else:
-        return obj
-
-
-def is_union_type(type_hint: type) -> bool:
-    return type(type_hint) is UnionType
-
-
-def get_inner_type(type_hint: type) -> type:
-    if is_union_type(type_hint):
-        # Return the first non-None type
-        return next(t for t in get_args(type_hint) if t is not type(None))
-    return type_hint
-
-
-def from_dict(t: type, data: dict[str, Any] | None) -> Any:
-    """
-    Dynamically instantiate a data class from a dictionary, handling nested data classes.
-    """
-    if not data:
-        return None
-
-    try:
-        # Attempt to create an instance of the data_class
-        field_values = {}
-        for field in fields(t):
-            field_value = data.get(field.name)
-            field_type = get_inner_type(field.type)
-            if field_value is not None:
-                # If the field is another dataclass, recursively instantiate it
-                if is_dataclass(field_type):
-                    field_value = from_dict(field_type, field_value)
-                elif isinstance(field_type, Path | str) and isinstance(
-                    field_value, str
-                ):
-                    field_value = (
-                        Path(field_value) if field_type == Path else field_value
-                    )
-
-            if (
-                field.default is not dataclasses.MISSING
-                or field.default_factory is not dataclasses.MISSING
-            ):
-                # Field has a default value. We cannot set the value to None
-                if field_value is not None:
-                    field_values[field.name] = field_value
-            else:
-                field_values[field.name] = field_value
-
-        return t(**field_values)
-
-    except (TypeError, ValueError) as e:
-        print(f"Failed to instantiate {t.__name__}: {e}")
-        return None
--- a/pkgs/clan-app/default.nix
+++ b/pkgs/clan-app/default.nix
@@ -94,7 +94,14 @@ python3.pkgs.buildPythonApplication rec {
  # that all necessary dependencies are consistently available both
  # at build time and runtime,
  buildInputs = allPythonDeps ++ runtimeDependencies;
-  propagatedBuildInputs = allPythonDeps ++ runtimeDependencies;
+  propagatedBuildInputs =
+    allPythonDeps
+    ++ runtimeDependencies
+    ++ [
+
+      # TODO: see postFixup clan-cli/default.nix:L188
+      clan-cli.propagatedBuildInputs
+    ];

  # also re-expose dependencies so we test them in CI
  passthru = {
--- a/pkgs/clan-app/flake-module.nix
+++ b/pkgs/clan-app/flake-module.nix
@@ -14,7 +14,7 @@
    else
      {
        devShells.clan-app = pkgs.callPackage ./shell.nix {
-          inherit (config.packages) clan-app webview-ui;
+          inherit (config.packages) clan-app;
          inherit self';
        };
        packages.clan-app = pkgs.python3.pkgs.callPackage ./default.nix {
--- a/pkgs/clan-app/shell.nix
+++ b/pkgs/clan-app/shell.nix
@@ -12,7 +12,6 @@
  python3,
  gtk4,
  libadwaita,
-  webview-ui,
  self',
 }:

@@ -29,7 +28,7 @@ let
    ]);
 in
 mkShell {
-  inherit (clan-app) nativeBuildInputs;
+  inherit (clan-app) nativeBuildInputs propagatedBuildInputs;

  inputsFrom = [ self'.devShells.default ];

@@ -67,8 +66,5 @@ mkShell {
    export XDG_DATA_DIRS=${gtk4}/share/gsettings-schemas/gtk4-4.14.4:$XDG_DATA_DIRS
    export XDG_DATA_DIRS=${gsettings-desktop-schemas}/share/gsettings-schemas/gsettings-desktop-schemas-46.0:$XDG_DATA_DIRS

-    # Add the webview-ui to the .webui directory
-    ln -nsf ${webview-ui}/lib/node_modules/@clan/webview-ui/dist/ ./clan_app/.webui
-
  '';
 }
--- a/pkgs/clan-cli/clan_cli/init.py
+++ b/pkgs/clan-cli/clan_cli/init.py
@@ -7,10 +7,10 @@ from types import ModuleType
 # These imports are unused, but necessary for @API.register to run once.
 from clan_cli.api import directory, mdns_discovery, modules
 from clan_cli.arg_actions import AppendOptionAction
-from clan_cli.clan import show
+from clan_cli.clan import show, update

 # API endpoints that are not used in the cli.
-__all__ = ["directory", "mdns_discovery", "modules"]
+__all__ = ["directory", "mdns_discovery", "modules", "update"]

 from . import (
    backups,
--- a/pkgs/clan-cli/clan_cli/api/init.py
+++ b/pkgs/clan-cli/clan_cli/api/init.py
@@ -1,141 +1,22 @@
-import dataclasses
-import json
 from collections.abc import Callable
-from dataclasses import dataclass, fields, is_dataclass
+from dataclasses import dataclass
 from functools import wraps
 from inspect import Parameter, Signature, signature
-from pathlib import Path
-from types import UnionType
 from typing import (
    Annotated,
    Any,
    Generic,
    Literal,
    TypeVar,
-    get_args,
-    get_origin,
    get_type_hints,
 )

+from .serde import dataclass_to_dict, from_dict, sanitize_string
+
+__all__ = ["from_dict", "dataclass_to_dict", "sanitize_string"]
+
 from clan_cli.errors import ClanError

-
-def sanitize_string(s: str) -> str:
-    # Using the native string sanitizer to handle all edge cases
-    # Remove the outer quotes '"string"'
-    return json.dumps(s)[1:-1]
-
-
-def dataclass_to_dict(obj: Any) -> Any:
-    """
-    Utility function to convert dataclasses to dictionaries
-    It converts all nested dataclasses, lists, tuples, and dictionaries to dictionaries
-
-    It does NOT convert member functions.
-    """
-    if is_dataclass(obj):
-        return {
-            # Use either the original name or name
-            sanitize_string(
-                field.metadata.get("original_name", field.name)
-            ): dataclass_to_dict(getattr(obj, field.name))
-            for field in fields(obj)  # type: ignore
-        }
-    elif isinstance(obj, list | tuple):
-        return [dataclass_to_dict(item) for item in obj]
-    elif isinstance(obj, dict):
-        return {sanitize_string(k): dataclass_to_dict(v) for k, v in obj.items()}
-    elif isinstance(obj, Path):
-        return sanitize_string(str(obj))
-    elif isinstance(obj, str):
-        return sanitize_string(obj)
-    else:
-        return obj
-
-
-def is_union_type(type_hint: type) -> bool:
-    return type(type_hint) is UnionType
-
-
-def get_inner_type(type_hint: type) -> type:
-    if is_union_type(type_hint):
-        # Return the first non-None type
-        return next(t for t in get_args(type_hint) if t is not type(None))
-    return type_hint
-
-
-def get_second_type(type_hint: type[dict]) -> type:
-    """
-    Get the value type of a dictionary type hint
-    """
-    args = get_args(type_hint)
-    if len(args) == 2:
-        # Return the second argument, which should be the value type (Machine)
-        return args[1]
-
-    raise ValueError(f"Invalid type hint for dict: {type_hint}")
-
-
-def from_dict(t: type, data: dict[str, Any] | None) -> Any:
-    """
-    Dynamically instantiate a data class from a dictionary, handling nested data classes.
-    """
-    if data is None:
-        return None
-
-    try:
-        # Attempt to create an instance of the data_class
-        field_values = {}
-        for field in fields(t):
-            original_name = field.metadata.get("original_name", field.name)
-
-            field_value = data.get(original_name)
-
-            field_type = get_inner_type(field.type)  # type: ignore
-
-            if original_name in data:
-                # If the field is another dataclass, recursively instantiate it
-                if is_dataclass(field_type):
-                    field_value = from_dict(field_type, field_value)
-                elif isinstance(field_type, Path | str) and isinstance(
-                    field_value, str
-                ):
-                    field_value = (
-                        Path(field_value) if field_type == Path else field_value
-                    )
-                elif get_origin(field_type) is dict and isinstance(field_value, dict):
-                    # The field is a dictionary with a specific type
-                    inner_type = get_second_type(field_type)
-                    field_value = {
-                        k: from_dict(inner_type, v) for k, v in field_value.items()
-                    }
-                elif get_origin is list and isinstance(field_value, list):
-                    # The field is a list with a specific type
-                    inner_type = get_args(field_type)[0]
-                    field_value = [from_dict(inner_type, v) for v in field_value]
-
-            # Set the value
-            if (
-                field.default is not dataclasses.MISSING
-                or field.default_factory is not dataclasses.MISSING
-            ):
-                # Fields with default value
-                # a: Int = 1
-                # b: list = Field(default_factory=list)
-                if original_name in data or field_value is not None:
-                    field_values[field.name] = field_value
-            else:
-                # Fields without default value
-                # a: Int
-                field_values[field.name] = field_value
-
-        return t(**field_values)
-
-    except (TypeError, ValueError) as e:
-        print(f"Failed to instantiate {t.__name__}: {e} {data}")
-        return None
-
-
 T = TypeVar("T")

 ResponseDataType = TypeVar("ResponseDataType")
--- a/pkgs/clan-cli/clan_cli/api/serde.py
+++ b/pkgs/clan-cli/clan_cli/api/serde.py
@@ -0,0 +1,106 @@
+"""
+This module provides utility functions for serialization and deserialization of data classes.
+
+Functions:
+- sanitize_string(s: str) -> str: Ensures a string is properly escaped for json serializing.
+- dataclass_to_dict(obj: Any) -> Any: Converts a data class and its nested data classes, lists, tuples, and dictionaries to dictionaries.
+- from_dict(t: type[T], data: Any) -> T: Dynamically instantiates a data class from a dictionary, constructing nested data classes, validates all required fields exist and have the expected type.
+
+Classes:
+- TypeAdapter: A Pydantic type adapter for data classes.
+
+Exceptions:
+- ValidationError: Raised when there is a validation error during deserialization.
+- ClanError: Raised when there is an error during serialization or deserialization.
+
+Dependencies:
+- dataclasses: Provides the @dataclass decorator and related functions for creating data classes.
+- json: Provides functions for working with JSON data.
+- collections.abc: Provides abstract base classes for collections.
+- functools: Provides functions for working with higher-order functions and decorators.
+- inspect: Provides functions for inspecting live objects.
+- operator: Provides functions for working with operators.
+- pathlib: Provides classes for working with filesystem paths.
+- types: Provides functions for working with types.
+- typing: Provides support for type hints.
+- pydantic: A library for data validation and settings management.
+- pydantic_core: Core functionality for Pydantic.
+
+Note: This module assumes the presence of other modules and classes such as `ClanError` and `ErrorDetails` from the `clan_cli.errors` module.
+"""
+
+import json
+from dataclasses import dataclass, fields, is_dataclass
+from pathlib import Path
+from typing import (
+    Any,
+    TypeVar,
+)
+
+from pydantic import TypeAdapter, ValidationError
+from pydantic_core import ErrorDetails
+
+from clan_cli.errors import ClanError
+
+
+def sanitize_string(s: str) -> str:
+    # Using the native string sanitizer to handle all edge cases
+    # Remove the outer quotes '"string"'
+    return json.dumps(s)[1:-1]
+
+
+def dataclass_to_dict(obj: Any, *, use_alias: bool = True) -> Any:
+    def _to_dict(obj: Any) -> Any:
+        """
+        Utility function to convert dataclasses to dictionaries
+        It converts all nested dataclasses, lists, tuples, and dictionaries to dictionaries
+
+        It does NOT convert member functions.
+        """
+        if is_dataclass(obj):
+            return {
+                # Use either the original name or name
+                sanitize_string(
+                    field.metadata.get("alias", field.name) if use_alias else field.name
+                ): _to_dict(getattr(obj, field.name))
+                for field in fields(obj)
+                if not field.name.startswith("_")  # type: ignore
+            }
+        elif isinstance(obj, list | tuple):
+            return [_to_dict(item) for item in obj]
+        elif isinstance(obj, dict):
+            return {sanitize_string(k): _to_dict(v) for k, v in obj.items()}
+        elif isinstance(obj, Path):
+            return sanitize_string(str(obj))
+        elif isinstance(obj, str):
+            return sanitize_string(obj)
+        else:
+            return obj
+
+    return _to_dict(obj)
+
+
+T = TypeVar("T", bound=dataclass)  # type: ignore
+
+
+def from_dict(t: type[T], data: Any) -> T:
+    """
+    Dynamically instantiate a data class from a dictionary, handling nested data classes.
+    We use dataclasses. But the deserialization logic of pydantic takes a lot of complexity.
+    """
+    adapter = TypeAdapter(t)
+    try:
+        return adapter.validate_python(
+            data,
+        )
+    except ValidationError as e:
+        fst_error: ErrorDetails = e.errors()[0]
+        if not fst_error:
+            raise ClanError(msg=str(e))
+
+        msg = fst_error.get("msg")
+        loc = fst_error.get("loc")
+        field_path = "Unknown"
+        if loc:
+            field_path = str(loc)
+        raise ClanError(msg=msg, location=f"{t!s}: {field_path}", description=str(e))
--- a/pkgs/clan-cli/clan_cli/api/util.py
+++ b/pkgs/clan-cli/clan_cli/api/util.py
@@ -74,7 +74,9 @@ def type_to_dict(t: Any, scope: str = "", type_map: dict[TypeVar, type] = {}) ->
    if dataclasses.is_dataclass(t):
        fields = dataclasses.fields(t)
        properties = {
-            f.name: type_to_dict(f.type, f"{scope} {t.__name__}.{f.name}", type_map)
+            f.metadata.get("alias", f.name): type_to_dict(
+                f.type, f"{scope} {t.__name__}.{f.name}", type_map
+            )
            for f in fields
            if not f.name.startswith("_")
        }
--- a/pkgs/clan-cli/clan_cli/clan/inspect.py
+++ b/pkgs/clan-cli/clan_cli/clan/inspect.py
@@ -14,7 +14,7 @@ from ..vms.inspect import VmConfig, inspect_vm

@dataclass
 class FlakeConfig:
-    flake_url: str | Path
+    flake_url: FlakeId
    flake_attr: str

    clan_name: str
@@ -89,7 +89,7 @@ def inspect_flake(flake_url: str | Path, machine_name: str) -> FlakeConfig:
    meta = nix_metadata(flake_url)
    return FlakeConfig(
        vm=vm,
-        flake_url=flake_url,
+        flake_url=FlakeId(flake_url),
        clan_name=clan_name,
        flake_attr=machine_name,
        nar_hash=meta["locked"]["narHash"],
--- a/pkgs/clan-cli/clan_cli/history/add.py
+++ b/pkgs/clan-cli/clan_cli/history/add.py
@@ -62,7 +62,7 @@ def list_history() -> list[HistoryEntry]:

 def new_history_entry(url: str, machine: str) -> HistoryEntry:
    flake = inspect_flake(url, machine)
-    flake.flake_url = str(flake.flake_url)
+    flake.flake_url = flake.flake_url
    return HistoryEntry(
        flake=flake,
        last_used=datetime.datetime.now().isoformat(),
--- a/pkgs/clan-cli/clan_cli/history/update.py
+++ b/pkgs/clan-cli/clan_cli/history/update.py
@@ -16,7 +16,7 @@ def update_history() -> list[HistoryEntry]:

    for entry in logs:
        try:
-            meta = nix_metadata(entry.flake.flake_url)
+            meta = nix_metadata(str(entry.flake.flake_url))
        except ClanCmdError as e:
            print(f"Failed to update {entry.flake.flake_url}: {e}")
            continue
@@ -31,7 +31,7 @@ def update_history() -> list[HistoryEntry]:
                machine_name=entry.flake.flake_attr,
            )
            flake = inspect_flake(uri.get_url(), uri.machine_name)
-            flake.flake_url = str(flake.flake_url)
+            flake.flake_url = flake.flake_url
            entry = HistoryEntry(
                flake=flake, last_used=datetime.datetime.now().isoformat()
            )
--- a/pkgs/clan-cli/clan_cli/inventory/classes.py
+++ b/pkgs/clan-cli/clan_cli/inventory/classes.py
@@ -153,7 +153,7 @@ class ServiceSingleDisk:
 class Service:
    borgbackup: dict[str, ServiceBorgbackup] = field(default_factory = dict)
    packages: dict[str, ServicePackage] = field(default_factory = dict)
-    single_disk: dict[str, ServiceSingleDisk] = field(default_factory = dict, metadata = {"original_name": "single-disk"})
+    single_disk: dict[str, ServiceSingleDisk] = field(default_factory = dict, metadata = {"alias": "single-disk"})


@dataclass
--- a/pkgs/clan-cli/clan_cli/machines/update.py
+++ b/pkgs/clan-cli/clan_cli/machines/update.py
@@ -13,6 +13,7 @@ from ..facts.upload import upload_secrets
 from ..machines.machines import Machine
 from ..nix import nix_command, nix_metadata
 from ..ssh import HostKeyCheck
+from ..vars.generate import generate_vars
 from .inventory import get_all_machines, get_selected_machines
 from .machine_group import MachineGroup

@@ -93,6 +94,7 @@ def deploy_machine(machines: MachineGroup) -> None:
        env["NIX_SSHOPTS"] = ssh_arg

        generate_facts([machine], None, False)
+        generate_vars([machine], None, False)
        upload_secrets(machine)

        path = upload_sources(".", target)
--- a/pkgs/clan-cli/default.nix
+++ b/pkgs/clan-cli/default.nix
@@ -17,6 +17,8 @@
  setuptools,
  stdenv,

+  pydantic,
+
  # custom args
  clan-core-path,
  nixpkgs,
@@ -28,6 +30,7 @@
 let
  pythonDependencies = [
    argcomplete # Enables shell completions
+    pydantic # Dataclass deserialisation / validation / schemas
  ];

  # load nixpkgs runtime dependencies from a json file
@@ -181,6 +184,7 @@ python3.pkgs.buildPythonApplication {
  '';

  # Clean up after the package to avoid leaking python packages into a devshell
+  # TODO: factor seperate cli / API packages
  postFixup = ''
    rm $out/nix-support/propagated-build-inputs
  '';
--- a/pkgs/clan-cli/flake-module.nix
+++ b/pkgs/clan-cli/flake-module.nix
@@ -62,7 +62,12 @@
          name = "clan-cli-docs";
          src = ./.;

-          buildInputs = [ pkgs.python3 ];
+          buildInputs = [
+
+            # TODO: see postFixup clan-cli/default.nix:L188
+            pkgs.python3
+            self'.packages.clan-cli.propagatedBuildInputs
+          ];

          installPhase = ''
            ${self'.packages.classgen}/bin/classgen ${self'.packages.inventory-schema}/schema.json ./clan_cli/inventory/classes.py
@@ -77,7 +82,12 @@
          name = "clan-ts-api";
          src = ./.;

-          buildInputs = [ pkgs.python3 ];
+          buildInputs = [
+            pkgs.python3
+
+            # TODO: see postFixup clan-cli/default.nix:L188
+            self'.packages.clan-cli.propagatedBuildInputs
+          ];

          installPhase = ''
            ${self'.packages.classgen}/bin/classgen ${self'.packages.inventory-schema}/schema.json ./clan_cli/inventory/classes.py
--- a/pkgs/clan-cli/tests/test_deserializers.py
+++ b/pkgs/clan-cli/tests/test_deserializers.py
@@ -0,0 +1,179 @@
+from dataclasses import dataclass, field
+from pathlib import Path
+
+import pytest
+
+# Functions to test
+from clan_cli.api import dataclass_to_dict, from_dict
+from clan_cli.errors import ClanError
+from clan_cli.inventory import (
+    Inventory,
+    Machine,
+    MachineDeploy,
+    Meta,
+    Service,
+    ServiceBorgbackup,
+    ServiceBorgbackupRole,
+    ServiceBorgbackupRoleClient,
+    ServiceBorgbackupRoleServer,
+    ServiceMeta,
+)
+
+
+def test_simple() -> None:
+    @dataclass
+    class Person:
+        name: str
+
+    person_dict = {
+        "name": "John",
+    }
+
+    expected_person = Person(
+        name="John",
+    )
+
+    assert from_dict(Person, person_dict) == expected_person
+
+
+def test_nested() -> None:
+    @dataclass
+    class Age:
+        value: str
+
+    @dataclass
+    class Person:
+        name: str
+        # deeply nested dataclasses
+        age: Age
+        age_list: list[Age]
+        age_dict: dict[str, Age]
+        # Optional field
+        home: Path | None
+
+    person_dict = {
+        "name": "John",
+        "age": {
+            "value": "99",
+        },
+        "age_list": [{"value": "66"}, {"value": "77"}],
+        "age_dict": {"now": {"value": "55"}, "max": {"value": "100"}},
+        "home": "/home",
+    }
+
+    expected_person = Person(
+        name="John",
+        age=Age("99"),
+        age_list=[Age("66"), Age("77")],
+        age_dict={"now": Age("55"), "max": Age("100")},
+        home=Path("/home"),
+    )
+
+    assert from_dict(Person, person_dict) == expected_person
+
+
+def test_simple_field_missing() -> None:
+    @dataclass
+    class Person:
+        name: str
+
+    person_dict = {}
+
+    with pytest.raises(ClanError):
+        from_dict(Person, person_dict)
+
+
+def test_deserialize_extensive_inventory() -> None:
+    # TODO: Make this an abstract test, so it doesn't break the test if the inventory changes
+    data = {
+        "meta": {"name": "superclan", "description": "nice clan"},
+        "services": {
+            "borgbackup": {
+                "instance1": {
+                    "meta": {
+                        "name": "borg1",
+                    },
+                    "roles": {
+                        "client": {},
+                        "server": {},
+                    },
+                }
+            },
+        },
+        "machines": {"foo": {"name": "foo", "deploy": {}}},
+    }
+    expected = Inventory(
+        meta=Meta(name="superclan", description="nice clan"),
+        services=Service(
+            borgbackup={
+                "instance1": ServiceBorgbackup(
+                    meta=ServiceMeta(name="borg1"),
+                    roles=ServiceBorgbackupRole(
+                        client=ServiceBorgbackupRoleClient(),
+                        server=ServiceBorgbackupRoleServer(),
+                    ),
+                )
+            }
+        ),
+        machines={"foo": Machine(deploy=MachineDeploy(), name="foo")},
+    )
+    assert from_dict(Inventory, data) == expected
+
+
+def test_alias_field() -> None:
+    @dataclass
+    class Person:
+        name: str = field(metadata={"alias": "--user-name--"})
+
+    data = {"--user-name--": "John"}
+    expected = Person(name="John")
+
+    person = from_dict(Person, data)
+
+    # Deserialize
+    assert person == expected
+
+    # Serialize with alias
+    assert dataclass_to_dict(person) == data
+
+    # Serialize without alias
+    assert dataclass_to_dict(person, use_alias=False) == {"name": "John"}
+
+
+def test_alias_field_from_orig_name() -> None:
+    """
+    Field declares an alias. But the data is provided with the field name.
+    """
+
+    @dataclass
+    class Person:
+        name: str = field(metadata={"alias": "--user-name--"})
+
+    data = {"user": "John"}
+
+    with pytest.raises(ClanError):
+        from_dict(Person, data)
+
+
+def test_path_field() -> None:
+    @dataclass
+    class Person:
+        name: Path
+
+    data = {"name": "John"}
+    expected = Person(name=Path("John"))
+
+    assert from_dict(Person, data) == expected
+
+
+def test_private_public_fields() -> None:
+    @dataclass
+    class Person:
+        name: Path
+        _name: str | None = None
+
+    data = {"name": "John"}
+    expected = Person(name=Path("John"))
+    assert from_dict(Person, data) == expected
+
+    assert dataclass_to_dict(expected) == data
--- a/pkgs/clan-cli/tests/test_history_cli.py
+++ b/pkgs/clan-cli/tests/test_history_cli.py
@@ -27,7 +27,7 @@ def test_history_add(
    history_file = user_history_file()
    assert history_file.exists()
    history = [HistoryEntry(**entry) for entry in json.loads(open(history_file).read())]
-    assert history[0].flake.flake_url == str(test_flake_with_core.path)
+    assert str(history[0].flake.flake_url["loc"]) == str(test_flake_with_core.path)


@pytest.mark.impure
--- a/pkgs/clan-cli/tests/test_serializers.py
+++ b/pkgs/clan-cli/tests/test_serializers.py
@@ -0,0 +1,106 @@
+from dataclasses import dataclass, field
+
+# Functions to test
+from clan_cli.api import (
+    dataclass_to_dict,
+    sanitize_string,
+)
+
+
+#
+def test_sanitize_string() -> None:
+    # Simple strings
+    assert sanitize_string("Hello World") == "Hello World"
+    assert sanitize_string("Hello\nWorld") == "Hello\\nWorld"
+    assert sanitize_string("Hello\tWorld") == "Hello\\tWorld"
+    assert sanitize_string("Hello\rWorld") == "Hello\\rWorld"
+    assert sanitize_string("Hello\fWorld") == "Hello\\fWorld"
+    assert sanitize_string("Hello\vWorld") == "Hello\\u000bWorld"
+    assert sanitize_string("Hello\bWorld") == "Hello\\bWorld"
+    assert sanitize_string("Hello\\World") == "Hello\\\\World"
+    assert sanitize_string('Hello"World') == 'Hello\\"World'
+    assert sanitize_string("Hello'World") == "Hello'World"
+    assert sanitize_string("Hello\0World") == "Hello\\u0000World"
+    # Console escape characters
+
+    assert sanitize_string("\033[1mBold\033[0m") == "\\u001b[1mBold\\u001b[0m"  # Red
+    assert sanitize_string("\033[31mRed\033[0m") == "\\u001b[31mRed\\u001b[0m"  # Blue
+    assert (
+        sanitize_string("\033[42mGreen\033[0m") == "\\u001b[42mGreen\\u001b[0m"
+    )  # Green
+    assert sanitize_string("\033[4mUnderline\033[0m") == "\\u001b[4mUnderline\\u001b[0m"
+    assert (
+        sanitize_string("\033[91m\033[1mBold Red\033[0m")
+        == "\\u001b[91m\\u001b[1mBold Red\\u001b[0m"
+    )
+
+
+def test_dataclass_to_dict() -> None:
+    @dataclass
+    class Person:
+        name: str
+        age: int
+
+    person = Person(name="John", age=25)
+    expected_dict = {"name": "John", "age": 25}
+    assert dataclass_to_dict(person) == expected_dict
+
+
+def test_dataclass_to_dict_nested() -> None:
+    @dataclass
+    class Address:
+        city: str = "afghanistan"
+        zip: str = "01234"
+
+    @dataclass
+    class Person:
+        name: str
+        age: int
+        address: Address = field(default_factory=Address)
+
+    person1 = Person(name="John", age=25)
+    expected_dict1 = {
+        "name": "John",
+        "age": 25,
+        "address": {"city": "afghanistan", "zip": "01234"},
+    }
+    # address must be constructed with default values if not passed
+    assert dataclass_to_dict(person1) == expected_dict1
+
+    person2 = Person(name="John", age=25, address=Address(zip="0", city="Anywhere"))
+    expected_dict2 = {
+        "name": "John",
+        "age": 25,
+        "address": {"zip": "0", "city": "Anywhere"},
+    }
+    assert dataclass_to_dict(person2) == expected_dict2
+
+
+def test_dataclass_to_dict_defaults() -> None:
+    @dataclass
+    class Foo:
+        home: dict[str, str] = field(default_factory=dict)
+        work: list[str] = field(default_factory=list)
+
+    @dataclass
+    class Person:
+        name: str = field(default="jon")
+        age: int = field(default=1)
+        foo: Foo = field(default_factory=Foo)
+
+    default_person = Person()
+    expected_default = {
+        "name": "jon",
+        "age": 1,
+        "foo": {"home": {}, "work": []},
+    }
+    # address must be constructed with default values if not passed
+    assert dataclass_to_dict(default_person) == expected_default
+
+    real_person = Person(name="John", age=25, foo=Foo(home={"a": "b"}, work=["a", "b"]))
+    expected = {
+        "name": "John",
+        "age": 25,
+        "foo": {"home": {"a": "b"}, "work": ["a", "b"]},
+    }
+    assert dataclass_to_dict(real_person) == expected
--- a/pkgs/clan-vm-manager/clan_vm_manager/singletons/use_vms.py
+++ b/pkgs/clan-vm-manager/clan_vm_manager/singletons/use_vms.py
@@ -163,12 +163,12 @@ class ClanStore:
        del self.clan_store[str(vm.data.flake.flake_url)][vm.data.flake.flake_attr]

    def get_vm(self, uri: ClanURI) -> None | VMObject:
-        flake_id = Machine(uri.machine_name, uri.flake).get_id()
-        vm_store = self.clan_store.get(flake_id)
+        machine = Machine(uri.machine_name, uri.flake)
+        vm_store = self.clan_store.get(str(machine.flake))
        if vm_store is None:
            return None
-        machine = vm_store.get(uri.machine_name, None)
-        return machine
+        vm = vm_store.get(str(machine.name), None)
+        return vm

    def get_running_vms(self) -> list[VMObject]:
        return [
--- a/pkgs/clan-vm-manager/default.nix
+++ b/pkgs/clan-vm-manager/default.nix
@@ -39,7 +39,7 @@ let
    libadwaita
    webkitgtk_6_0
    adwaita-icon-theme
-  ];
+  ] ++ clan-cli.propagatedBuildInputs;

  # Deps including python packages from the local project
  allPythonDeps = [ (python3.pkgs.toPythonModule clan-cli) ] ++ externalPythonDeps;
@@ -84,7 +84,6 @@ python3.pkgs.buildPythonApplication rec {
    setuptools
    copyDesktopItems
    wrapGAppsHook
-
    gobject-introspection
  ];

@@ -93,7 +92,7 @@ python3.pkgs.buildPythonApplication rec {
  # that all necessary dependencies are consistently available both
  # at build time and runtime,
  buildInputs = allPythonDeps ++ runtimeDependencies;
-  propagatedBuildInputs = allPythonDeps ++ runtimeDependencies;
+  propagatedBuildInputs = allPythonDeps ++ runtimeDependencies ++ [ ];

  # also re-expose dependencies so we test them in CI
  passthru = {
--- a/pkgs/classgen/main.py
+++ b/pkgs/classgen/main.py
@@ -245,7 +245,7 @@ def generate_dataclass(schema: dict[str, Any], class_name: str = root_class) ->

        field_meta = None
        if field_name != prop:
-            field_meta = f"""{{"original_name": "{prop}"}}"""
+            field_meta = f"""{{"alias": "{prop}"}}"""

        finalize_field = partial(get_field_def, field_name, field_meta)

--- a/pkgs/webview-ui/app/eslint.config.mjs
+++ b/pkgs/webview-ui/app/eslint.config.mjs
@@ -1,9 +1,11 @@
 import eslint from "@eslint/js";
 import tseslint from "typescript-eslint";
 import tailwind from "eslint-plugin-tailwindcss";
+import pluginQuery from "@tanstack/eslint-plugin-query";

 export default tseslint.config(
  eslint.configs.recommended,
+  ...pluginQuery.configs["flat/recommended"],
  ...tseslint.configs.strict,
  ...tseslint.configs.stylistic,
  ...tailwind.configs["flat/recommended"],
--- a/pkgs/webview-ui/app/package-lock.json
+++ b/pkgs/webview-ui/app/package-lock.json
--- a/pkgs/webview-ui/app/package.json
+++ b/pkgs/webview-ui/app/package.json
@@ -38,8 +38,10 @@
    "vitest": "^1.6.0"
  },
  "dependencies": {
+    "@floating-ui/dom": "^1.6.8",
    "@modular-forms/solid": "^0.21.0",
    "@solid-primitives/storage": "^3.7.1",
+    "@tanstack/eslint-plugin-query": "^5.51.12",
    "@tanstack/solid-query": "^5.51.2",
    "material-icons": "^1.13.12",
    "nanoid": "^5.0.7",
--- a/pkgs/webview-ui/app/src/App.tsx
+++ b/pkgs/webview-ui/app/src/App.tsx
@@ -1,4 +1,4 @@
-import { createSignal, type Component } from "solid-js";
+import { createEffect, createSignal, type Component } from "solid-js";
 import { Layout } from "./layout/layout";
 import { Route, Router } from "./Routes";
 import { Toaster } from "solid-toast";
@@ -7,9 +7,20 @@ import { makePersisted } from "@solid-primitives/storage";

 // Some global state
 const [route, setRoute] = createSignal<Route>("machines");
+createEffect(() => {
+  console.log(route());
+});
+
 export { route, setRoute };

-const [activeURI, setActiveURI] = createSignal<string | null>(null);
+const [activeURI, setActiveURI] = makePersisted(
+  createSignal<string | null>(null),
+  {
+    name: "activeURI",
+    storage: localStorage,
+  }
+);
+
 export { activeURI, setActiveURI };

 const [clanList, setClanList] = makePersisted(createSignal<string[]>([]), {
@@ -17,8 +28,6 @@ const [clanList, setClanList] = makePersisted(createSignal<string[]>([]), {
  storage: localStorage,
 });

-clanList() && setActiveURI(clanList()[0]);
-
 export { clanList, setClanList };

 const App: Component = () => {
--- a/pkgs/webview-ui/app/src/Routes.tsx
+++ b/pkgs/webview-ui/app/src/Routes.tsx
@@ -9,6 +9,7 @@ import { Settings } from "./routes/settings";
 import { Welcome } from "./routes/welcome";
 import { Deploy } from "./routes/deploy";
 import { CreateMachine } from "./routes/machines/create";
+import { DiskView } from "./routes/disk/view";

 export type Route = keyof typeof routes;

@@ -63,6 +64,11 @@ export const routes = {
    label: "deploy",
    icon: "content_copy",
  },
+  diskConfig: {
+    child: DiskView,
+    label: "diskConfig",
+    icon: "disk",
+  },
 };

 interface RouterProps {
--- a/pkgs/webview-ui/app/src/floating/index.tsx
+++ b/pkgs/webview-ui/app/src/floating/index.tsx
@@ -0,0 +1,128 @@
+import { createEffect, createMemo, createSignal, onCleanup } from "solid-js";
+import type {
+  ComputePositionConfig,
+  ComputePositionReturn,
+  ReferenceElement,
+} from "@floating-ui/dom";
+import { computePosition } from "@floating-ui/dom";
+
+export interface UseFloatingOptions<
+  R extends ReferenceElement,
+  F extends HTMLElement,
+> extends Partial<ComputePositionConfig> {
+  whileElementsMounted?: (
+    reference: R,
+    floating: F,
+    update: () => void
+    // eslint-disable-next-line @typescript-eslint/no-invalid-void-type
+  ) => void | (() => void);
+}
+
+interface UseFloatingState extends Omit<ComputePositionReturn, "x" | "y"> {
+  x?: number | null;
+  y?: number | null;
+}
+
+export interface UseFloatingResult extends UseFloatingState {
+  update(): void;
+}
+
+export function useFloating<R extends ReferenceElement, F extends HTMLElement>(
+  reference: () => R | undefined | null,
+  floating: () => F | undefined | null,
+  options?: UseFloatingOptions<R, F>
+): UseFloatingResult {
+  const placement = () => options?.placement ?? "bottom";
+  const strategy = () => options?.strategy ?? "absolute";
+
+  const [data, setData] = createSignal<UseFloatingState>({
+    x: null,
+    y: null,
+    placement: placement(),
+    strategy: strategy(),
+    middlewareData: {},
+  });
+
+  const [error, setError] = createSignal<{ value: unknown } | undefined>();
+
+  createEffect(() => {
+    const currentError = error();
+    if (currentError) {
+      throw currentError.value;
+    }
+  });
+
+  const version = createMemo(() => {
+    reference();
+    floating();
+    return {};
+  });
+
+  function update() {
+    const currentReference = reference();
+    const currentFloating = floating();
+
+    if (currentReference && currentFloating) {
+      const capturedVersion = version();
+      computePosition(currentReference, currentFloating, {
+        middleware: options?.middleware,
+        placement: placement(),
+        strategy: strategy(),
+      }).then(
+        (currentData) => {
+          // Check if it's still valid
+          if (capturedVersion === version()) {
+            setData(currentData);
+          }
+        },
+        (err) => {
+          setError(err);
+        }
+      );
+    }
+  }
+
+  createEffect(() => {
+    const currentReference = reference();
+    const currentFloating = floating();
+
+    options?.middleware;
+    placement();
+    strategy();
+
+    if (currentReference && currentFloating) {
+      if (options?.whileElementsMounted) {
+        const cleanup = options.whileElementsMounted(
+          currentReference,
+          currentFloating,
+          update
+        );
+
+        if (cleanup) {
+          onCleanup(cleanup);
+        }
+      } else {
+        update();
+      }
+    }
+  });
+
+  return {
+    get x() {
+      return data().x;
+    },
+    get y() {
+      return data().y;
+    },
+    get placement() {
+      return data().placement;
+    },
+    get strategy() {
+      return data().strategy;
+    },
+    get middlewareData() {
+      return data().middlewareData;
+    },
+    update,
+  };
+}
--- a/pkgs/webview-ui/app/src/layout/header.tsx
+++ b/pkgs/webview-ui/app/src/layout/header.tsx
@@ -1,15 +1,20 @@
 import { createQuery } from "@tanstack/solid-query";
 import { activeURI, setRoute } from "../App";
 import { callApi } from "../api";
-import { Show } from "solid-js";
+import { Accessor, createEffect, Show } from "solid-js";

-export const Header = () => {
-  const { isLoading, data } = createQuery(() => ({
-    queryKey: [`${activeURI()}:meta`],
+interface HeaderProps {
+  clan_dir: Accessor<string | null>;
+}
+export const Header = (props: HeaderProps) => {
+  const { clan_dir } = props;
+
+  const query = createQuery(() => ({
+    queryKey: [clan_dir(), "meta"],
    queryFn: async () => {
-      const currUri = activeURI();
-      if (currUri) {
-        const result = await callApi("show_clan_meta", { uri: currUri });
+      const curr = clan_dir();
+      if (curr) {
+        const result = await callApi("show_clan_meta", { uri: curr });
        if (result.status === "error") throw new Error("Failed to fetch data");
        return result.data;
      }
@@ -29,16 +34,25 @@ export const Header = () => {
        </span>
      </div>
      <div class="flex-1">
-        <div class="tooltip tooltip-right" data-tip={data?.name || activeURI()}>
-          <div class="avatar placeholder online mx-4">
-            <div class="w-10 rounded-full bg-slate-700 text-neutral-content">
-              <span class="text-xl">C</span>
-              <Show when={data?.name}>
-                {(name) => <span class="text-xl">{name()}</span>}
-              </Show>
+        <Show when={!query.isFetching && query.data}>
+          {(meta) => (
+            <div class="tooltip tooltip-right" data-tip={activeURI()}>
+              <div class="avatar placeholder online mx-4">
+                <div class="w-10 rounded-full bg-slate-700 text-3xl text-neutral-content">
+                  {meta().name.slice(0, 1)}
+                </div>
+              </div>
            </div>
-          </div>
-        </div>
+          )}
+        </Show>
+        <span class="flex flex-col">
+          <Show when={!query.isFetching && query.data}>
+            {(meta) => [
+              <span class="text-primary">{meta().name}</span>,
+              <span class="text-neutral">{meta()?.description}</span>,
+            ]}
+          </Show>
+        </span>
      </div>
      <div class="flex-none">
        <span class="tooltip tooltip-bottom" data-tip="Settings">
--- a/pkgs/webview-ui/app/src/layout/layout.tsx
+++ b/pkgs/webview-ui/app/src/layout/layout.tsx
@@ -1,7 +1,7 @@
 import { Component, JSXElement, Show } from "solid-js";
 import { Header } from "./header";
 import { Sidebar } from "../Sidebar";
-import { clanList, route, setRoute } from "../App";
+import { activeURI, clanList, route, setRoute } from "../App";

 interface LayoutProps {
  children: JSXElement;
@@ -18,7 +18,7 @@ export const Layout: Component<LayoutProps> = (props) => {
        />
        <div class="drawer-content">
          <Show when={route() !== "welcome"}>
-            <Header />
+            <Header clan_dir={activeURI} />
          </Show>
          {props.children}
        </div>
--- a/pkgs/webview-ui/app/src/routes/clan/editClan.tsx
+++ b/pkgs/webview-ui/app/src/routes/clan/editClan.tsx
@@ -0,0 +1,175 @@
+import { OperationResponse, callApi, pyApi } from "@/src/api";
+import { Accessor, Show, Switch, Match } from "solid-js";
+import {
+  SubmitHandler,
+  createForm,
+  required,
+  reset,
+} from "@modular-forms/solid";
+import toast from "solid-toast";
+import { createQuery } from "@tanstack/solid-query";
+
+type CreateForm = Meta;
+
+interface EditClanFormProps {
+  directory: Accessor<string>;
+  done: () => void;
+}
+export const EditClanForm = (props: EditClanFormProps) => {
+  const { directory } = props;
+  const details = createQuery(() => ({
+    queryKey: [directory(), "meta"],
+    queryFn: async () => {
+      const result = await callApi("show_clan_meta", { uri: directory() });
+      if (result.status === "error") throw new Error("Failed to fetch data");
+      return result.data;
+    },
+  }));
+
+  return (
+    <Switch>
+      <Match when={details?.data}>
+        {(data) => (
+          <FinalEditClanForm
+            initial={data()}
+            directory={directory()}
+            done={props.done}
+          />
+        )}
+      </Match>
+    </Switch>
+  );
+};
+
+interface FinalEditClanFormProps {
+  initial: CreateForm;
+  directory: string;
+  done: () => void;
+}
+export const FinalEditClanForm = (props: FinalEditClanFormProps) => {
+  const [formStore, { Form, Field }] = createForm<CreateForm>({
+    initialValues: props.initial,
+  });
+
+  const handleSubmit: SubmitHandler<CreateForm> = async (values, event) => {
+    await toast.promise(
+      (async () => {
+        await callApi("update_clan_meta", {
+          options: {
+            directory: props.directory,
+            meta: values,
+          },
+        });
+      })(),
+      {
+        loading: "Updating clan...",
+        success: "Clan Successfully updated",
+        error: "Failed to update clan",
+      }
+    );
+    props.done();
+  };
+
+  return (
+    <div class="card card-normal">
+      <Form onSubmit={handleSubmit} shouldActive>
+        <Field name="icon">
+          {(field, props) => (
+            <>
+              <figure>
+                <Show
+                  when={field.value}
+                  fallback={
+                    <span class="material-icons aspect-square size-60 rounded-lg text-[18rem]">
+                      group
+                    </span>
+                  }
+                >
+                  {(icon) => (
+                    <img
+                      class="aspect-square size-60 rounded-lg"
+                      src={icon()}
+                      alt="Clan Logo"
+                    />
+                  )}
+                </Show>
+              </figure>
+            </>
+          )}
+        </Field>
+        <div class="card-body">
+          <Field
+            name="name"
+            validate={[required("Please enter a unique name for the clan.")]}
+          >
+            {(field, props) => (
+              <label class="form-control w-full">
+                <div class="label">
+                  <span class="label-text block after:ml-0.5 after:text-primary after:content-['*']">
+                    Name
+                  </span>
+                </div>
+
+                <input
+                  {...props}
+                  disabled={formStore.submitting}
+                  required
+                  placeholder="Clan Name"
+                  class="input input-bordered"
+                  classList={{ "input-error": !!field.error }}
+                  value={field.value}
+                />
+                <div class="label">
+                  {field.error && (
+                    <span class="label-text-alt">{field.error}</span>
+                  )}
+                </div>
+              </label>
+            )}
+          </Field>
+          <Field name="description">
+            {(field, props) => (
+              <label class="form-control w-full">
+                <div class="label">
+                  <span class="label-text">Description</span>
+                </div>
+
+                <input
+                  {...props}
+                  disabled={formStore.submitting}
+                  required
+                  type="text"
+                  placeholder="Some words about your clan"
+                  class="input input-bordered"
+                  classList={{ "input-error": !!field.error }}
+                  value={field.value || ""}
+                />
+                <div class="label">
+                  {field.error && (
+                    <span class="label-text-alt">{field.error}</span>
+                  )}
+                </div>
+              </label>
+            )}
+          </Field>
+          {
+            <div class="card-actions justify-end">
+              <button
+                class="btn btn-primary"
+                type="submit"
+                disabled={formStore.submitting}
+              >
+                Save
+              </button>
+            </div>
+          }
+        </div>
+      </Form>
+    </div>
+  );
+};
+
+type Meta = Extract<
+  OperationResponse<"show_clan_meta">,
+  { status: "success" }
+>["data"];
--- a/pkgs/webview-ui/app/src/routes/disk/view.tsx
+++ b/pkgs/webview-ui/app/src/routes/disk/view.tsx
@@ -0,0 +1,33 @@
+import { callApi } from "@/src/api";
+import { activeURI } from "@/src/App";
+import { createQuery } from "@tanstack/solid-query";
+import { createEffect } from "solid-js";
+import toast from "solid-toast";
+
+export function DiskView() {
+  const query = createQuery(() => ({
+    queryKey: ["disk", activeURI()],
+    queryFn: async () => {
+      const currUri = activeURI();
+      if (currUri) {
+        // Example of calling an API
+        const result = await callApi("get_inventory", { base_path: currUri });
+        if (result.status === "error") throw new Error("Failed to fetch data");
+        return result.data;
+      }
+    },
+  }));
+  createEffect(() => {
+    // Example debugging the data
+    console.log(query);
+  });
+  return (
+    <div>
+      <h1>Configure Disk</h1>
+      <p>
+        Select machine then configure the disk. Required before installing for
+        the first time.
+      </p>
+    </div>
+  );
+}
--- a/pkgs/webview-ui/app/src/routes/settings/index.tsx
+++ b/pkgs/webview-ui/app/src/routes/settings/index.tsx
@@ -12,8 +12,19 @@ import {
  setRoute,
  clanList,
 } from "@/src/App";
-import { For, Show } from "solid-js";
+import {
+  createEffect,
+  createSignal,
+  For,
+  Match,
+  Setter,
+  Show,
+  Switch,
+} from "solid-js";
 import { createQuery } from "@tanstack/solid-query";
+import { useFloating } from "@/src/floating";
+import { autoUpdate, flip, hide, offset, shift } from "@floating-ui/dom";
+import { EditClanForm } from "../clan/editClan";

 export const registerClan = async () => {
  try {
@@ -41,9 +52,10 @@ export const registerClan = async () => {

 interface ClanDetailsProps {
  clan_dir: string;
+  setEditURI: Setter<string | null>;
 }
 const ClanDetails = (props: ClanDetailsProps) => {
-  const { clan_dir } = props;
+  const { clan_dir, setEditURI } = props;

  const details = createQuery(() => ({
    queryKey: [clan_dir, "meta"],
@@ -54,10 +66,41 @@ const ClanDetails = (props: ClanDetailsProps) => {
    },
  }));

+  const [reference, setReference] = createSignal<HTMLElement>();
+  const [floating, setFloating] = createSignal<HTMLElement>();
+
+  // `position` is a reactive object.
+  const position = useFloating(reference, floating, {
+    placement: "top",
+
+    // pass options. Ensure the cleanup function is returned.
+    whileElementsMounted: (reference, floating, update) =>
+      autoUpdate(reference, floating, update, {
+        animationFrame: true,
+      }),
+    middleware: [
+      offset(5),
+      shift(),
+      flip(),
+
+      hide({
+        strategy: "referenceHidden",
+      }),
+    ],
+  });
+
  return (
    <div class="stat">
      <div class="stat-figure text-primary">
        <div class="join">
+          <button
+            class=" join-item btn-sm"
+            onClick={() => {
+              setEditURI(clan_dir);
+            }}
+          >
+            <span class="material-icons">edit</span>
+          </button>
          <button
            class=" join-item btn-sm"
            classList={{
@@ -72,75 +115,96 @@ const ClanDetails = (props: ClanDetailsProps) => {
            {activeURI() === clan_dir ? "active" : "select"}
          </button>
          <button
+            popovertarget={`clan-delete-popover-${clan_dir}`}
+            popovertargetaction="toggle"
+            ref={setReference}
            class="btn btn-ghost btn-outline join-item btn-sm"
-            onClick={() => {
-              setClanList((s) =>
-                s.filter((v, idx) => {
-                  if (v == clan_dir) {
-                    setActiveURI(
-                      clanList()[idx - 1] || clanList()[idx + 1] || null
-                    );
-                    return false;
-                  }
-                  return true;
-                })
-              );
-            }}
          >
            Remove
          </button>
+          <div
+            popover="auto"
+            id={`clan-delete-popover-${clan_dir}`}
+            ref={setFloating}
+            style={{
+              position: position.strategy,
+              top: `${position.y ?? 0}px`,
+              left: `${position.x ?? 0}px`,
+            }}
+            class="bg-transparent"
+          >
+            <button
+              class="btn btn-warning btn-sm"
+              onClick={() => {
+                setClanList((s) =>
+                  s.filter((v, idx) => {
+                    if (v == clan_dir) {
+                      setActiveURI(
+                        clanList()[idx - 1] || clanList()[idx + 1] || null
+                      );
+                      return false;
+                    }
+                    return true;
+                  })
+                );
+              }}
+            >
+              Remove from App
+            </button>
+          </div>
        </div>
      </div>
-      <div class="stat-title">Clan URI</div>
+      <div class="stat-title">{clan_dir}</div>

      <Show when={details.isSuccess}>
-        <div
-          class="stat-value"
-          // classList={{
-          //   "text-primary": activeURI() === clan_dir,
-          // }}
-        >
-          {details.data?.name}
-        </div>
+        <div class="stat-value">{details.data?.name}</div>
      </Show>
-      <Show
-        when={details.isSuccess && details.data?.description}
-        fallback={<div class="stat-desc text-lg">{clan_dir}</div>}
-      >
-        <div
-          class="stat-desc text-lg"
-          // classList={{
-          //   "text-primary": activeURI() === clan_dir,
-          // }}
-        >
-          {details.data?.description}
-        </div>
+      <Show when={details.isSuccess && details.data?.description}>
+        <div class="stat-desc text-lg">{details.data?.description}</div>
      </Show>
    </div>
  );
 };

 export const Settings = () => {
+  const [editURI, setEditURI] = createSignal<string | null>(null);
+
  return (
    <div class="card card-normal">
-      <div class="card-body">
-        <div class="label">
-          <div class="label-text">Registered Clans</div>
-          <button
-            class="btn btn-square btn-primary"
-            onClick={() => {
-              registerClan();
-            }}
-          >
-            <span class="material-icons">add</span>
-          </button>
-        </div>
-        <div class="stats stats-vertical shadow">
-          <For each={clanList()}>
-            {(value) => <ClanDetails clan_dir={value} />}
-          </For>
-        </div>
-      </div>
+      <Switch>
+        <Match when={editURI()}>
+          {(uri) => (
+            <EditClanForm
+              directory={uri}
+              done={() => {
+                setEditURI(null);
+              }}
+            />
+          )}
+        </Match>
+        <Match when={!editURI()}>
+          <div class="card-body">
+            <div class="label">
+              <div class="label-text">Registered Clans</div>
+              <button
+                class="btn btn-square btn-primary"
+                onClick={() => {
+                  registerClan();
+                }}
+              >
+                <span class="material-icons">add</span>
+              </button>
+            </div>
+            <div class="stats stats-vertical shadow">
+              <For each={clanList()}>
+                {(value) => (
+                  <ClanDetails clan_dir={value} setEditURI={setEditURI} />
+                )}
+              </For>
+            </div>
+          </div>
+        </Match>
+      </Switch>
    </div>
  );
 };
--- a/pkgs/webview-ui/flake-module.nix
+++ b/pkgs/webview-ui/flake-module.nix
@@ -16,7 +16,7 @@

        npmDeps = pkgs.fetchNpmDeps {
          src = ./app;
-          hash = "sha256-/PFSBAIodZjInElYoNsDQUV4isxmcvL3YM1hzAmdDWA=";
+          hash = "sha256-n9IXcfCpydykoYD+P/YNtNIwrvgJTZND0kg7oXBfmJ0=";
        };
        # The prepack script runs the build script, which we'd rather do in the build phase.
        npmPackFlags = [ "--ignore-scripts" ];