Commit Graph

622 Commits

Author SHA1 Message Date
Johannes Kirschbauer
4536a5b4f5 clan/flash: provide defaults for verbose flash options 2025-08-07 13:46:07 +02:00
Johannes Kirschbauer
a9cfda9acb dirs: add local path to clan_core flake in dirs 2025-08-07 13:46:07 +02:00
Mic92
273c83ec27 Merge pull request 'update/flake-upload: use ssh-ng:// for nix copy' (#4597) from local-build into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4597
2025-08-05 20:47:23 +00:00
Jörg Thalheim
44dcfa7844 rename --fetch-local to --upload-inputs 2025-08-05 19:31:29 +02:00
Jörg Thalheim
98f685f3ca update/flake-upload: set correct remote-program for macOS targets 2025-08-05 19:31:29 +02:00
Mic92
3fe65f1f12 Merge pull request 'machines update: support local build' (#4515) from local-build into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4515
2025-08-05 11:28:50 +00:00
Jörg Thalheim
6bb998f9dd update/flake-upload: use ssh-ng:// for nix copy
I had concurrency issues with `nix copy` and the ssh:// protocol when
using a machine both as the build host/target host (for different
machines), where it make the result path partially available to a
different update command thread.
2025-08-05 13:26:34 +02:00
DavHau
af7ce9b8ed machines update: support local build
Now the user can pass `--build-host local`, to select the local machine as a build host, in which case no ssh is used.

This means the admin machine does not necessarily have ssh set up to itself, which was confusing for many users.

Also this makes it easier to re-use a well configured nix remote build setup which is only available on the local machine. Eg if `--build-host local` nix' defaults for remote builds on that machine will be utilized.
2025-08-05 13:16:59 +02:00
DavHau
b74193514d ssh: refactor callers to use new Host interface 2025-08-05 13:16:59 +02:00
DavHau
c33fd4e504 ssh: Introduce LocalHost vs. Remote via Host interface
Motivation: local builds and deployments without ssh

Add a new interface `Host` which is implemented bei either `Remote` or `Localhost`

This simplifies all interactions with hosts. THe caller does ot need to know if the Host is remote or local in mot cases anymore
2025-08-05 13:16:59 +02:00
pinpox
f314eb04d6 Reduce targetHost warning level
The documentation currently lists setting targetHost in the NixOS
configuration as a slower, but valid option. Especially for dynamic
values, this is the recommended way but it results in a lot of annyoing
warnings.

This lowers the warning level, so it will only get printed on --debug
2025-08-05 11:52:06 +02:00
lassulus
0119fc06ca clan-cli select: show if select is cached or not 2025-08-02 21:26:39 +02:00
lassulus
5361261bd5 clan select: better error 2025-08-02 17:59:12 +02:00
lassulus
86e7bcc389 clan select: simplify select logging 2025-08-02 17:19:35 +02:00
Qubasa
0b05b0b1ec docs: review fixups
docs: review fixups

docs: review fixups

docs: fixup links in cli

docs: fixup links in cli
2025-08-01 14:53:31 +07:00
Qubasa
62a3503987 clan-lib: Always set a static private key for nixos-anywhere, to make --phases work properly 2025-07-31 17:06:00 +07:00
Mic92
d19ac1b9f5 Merge pull request 'machines update: refactor - simplify' (#4506) from simplify-update into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4506
2025-07-30 18:42:02 +00:00
a-kenji
62c4f735ed pkgs/clan/lib: Move get_clan_directories to dirs 2025-07-30 15:07:05 +02:00
Kenji Berthold
1791743444 Merge pull request 'pkgs/clan/lib: Add clan api to get the relative clan directory' (#4534) from kenji/ke-add-computed-directory-function into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4534
Reviewed-by: hsjobeki <hsjobeki@gmail.com>
2025-07-30 12:30:30 +00:00
a-kenji
c7ad875e7e pkgs/clan/lib: Add clan api to get the relative clan directory
This is a Continuation of: #4519
2025-07-30 14:22:23 +02:00
a-kenji
e167137672 pkgs/cli/lib: Remove obsolete clan creation function 2025-07-30 12:41:42 +02:00
DavHau
b30686269b machines update: fix lacks a signature by a trusted key
Despite using `root` as the ssh user, `ssh-ng` still fails with:

`error: cannot add path '/nix/store/...' because it lacks a signature by a trusted key
`

This does not happen with `ssh` instead of `ssh-ng`
2025-07-30 15:15:57 +07:00
Jörg Thalheim
1626d179a0 run_machine_update: document missing flag 2025-07-30 09:54:50 +02:00
Jörg Thalheim
092ac21dcd git_clan_flake_toplevel: gracefully handle permission errors
since this code is used as a default in the cli parser, we should not
crash on OSError because we wouldn't be abe to perform basic tasks such
as --help.
2025-07-30 09:54:50 +02:00
Jörg Thalheim
bd6f7b03af inline find_git_repo_root 2025-07-30 09:54:50 +02:00
Jörg Thalheim
de65619442 update: explain why we use sudo rather than --use-remote-sudo 2025-07-30 09:54:50 +02:00
DavHau
85dda9e125 machines update: refactor - simplify
This is an attempt to reduce cognitive load when trying to understand the host related logic in run_machine_update.

The change should not affect behavior.

Done:
- make it very clear early on, that build_host == target_host if build_host is not set.
- rename some variables to make relations more clear
- remove `deploy_host` variable. unnecessary complexity
- remove `become_root` variable. After simplifying via boolean algebra, this boils down to `build_host == target_host`.
2025-07-30 09:54:50 +02:00
Kenji Berthold
db579e169c Merge pull request 'pkgs/clan/lib: Fix directory functionality' (#4519) from kenji/ke-fix-directory-usage into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4519
2025-07-29 16:01:35 +00:00
a-kenji
31438d6781 pkgs/clan/lib: Fix directory functionality
Fix the directory functionality of clan (clan.directory).
The python API interface was not able to distinguish if the directory
was set to anything other than `self.src`.
Breaking every command that relied on the clan directory, for example:
- `clan machines update`
- `clan machines update-hardware-config`
See more in #2906

This is the first step in fixing all those commands.
Individual command support and implementation will be implemented in
follow ups.
2025-07-29 17:51:12 +02:00
a-kenji
7ef09343ed pkgs/clan/lib: Fix documentation of from_ssh_uri in the Remote class 2025-07-29 17:37:50 +02:00
hsjobeki
b421698f70 Merge pull request 'templates: fix urls for relative file paths' (#4520) from fix-template-urls into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4520
2025-07-29 15:19:20 +00:00
Johannes Kirschbauer
2776294de0 templates: url add support for home and abspath 2025-07-29 17:04:15 +02:00
Johannes Kirschbauer
7784df8180 templates: fix urls for relative file paths 2025-07-29 15:01:29 +02:00
Kenji Berthold
5d0ca5aff8 Merge pull request 'pkgs/clan/lib: Fix clan template creation when already in a flake' (#4501) from kenji/ke-clan-flakes-create-existing-flake-fix into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4501
2025-07-29 11:19:04 +00:00
DavHau
19a8101e98 Merge pull request 'pkgs/cli/create: Show less output by default' (#4499) from kenji/ke-create-show-less-output into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4499
2025-07-29 10:45:16 +00:00
DavHau
cb89fb97f1 clan machines update: add --fetch-local feature
Motivation: updating a machine fails, if it depends on a private github repo, as the remote will likely not be authenticated.

This adds a new flag `--fetch-local` to `clan machines update` which fetches all flake inputs prior to building, then uploads them to the build-host.

This also adds a new error message, when flake inputs could not fetched, to hint the user to use `--fetch-local`
2025-07-28 17:01:42 +07:00
Johannes Kirschbauer
98a0b9600b api/writability: add docstring 2025-07-27 12:47:37 +02:00
Johannes Kirschbauer
abeb517a22 api/writability: add unit test 2025-07-27 00:03:05 +02:00
Johannes Kirschbauer
fbdbcfa6d5 InventoryStore: factor write into _write for actual disk interaction 2025-07-27 00:02:47 +02:00
Johannes Kirschbauer
303af9af6b api: init get_machine_writeability 2025-07-27 00:01:51 +02:00
Johannes Kirschbauer
414e412e7e persist/writeability: expose is writeable key helper 2025-07-27 00:01:36 +02:00
Johannes Kirschbauer
c2e84f11af persist/util: add field helper 2025-07-27 00:01:07 +02:00
Johannes Kirschbauer
bf2eb000d5 api/set_machine: add unit tests 2025-07-26 23:59:51 +02:00
a-kenji
0a43721a45 pkgs/clan/lib: Fix clan template creation when already in a flake
Fix clan template creation when already in a flake.
Currently we already fail with very clear and descriptive error when
trying to evaluate the template of the flake we are in:
```
Failed to select template 'flake-parts' from flake '/tmp/superclan' (via attribute path: /tmp/superclan#clanInternals.templates.clan."flake-parts")
```

This is undesired behavior.
When we are trying to create a clan with `clan flakes create`.
We can't rely on the fact that the flake we are currently in exports flake templates.

Now we *try* to evaluate the flake we are in upon creation.
If there are no clan templates available, we now will fall back to
builtin templates.

Closes: #4472
2025-07-26 15:21:44 +02:00
a-kenji
51eb7bd0b5 pkgs/cli/create: Show less output by default
This masks the output of `nix flake update`, since it is quite verbose
and takes attention away to usually more interesting and pressing
information.

Example:
```
unpacking 'https://git.clan.lol/clan/clan-core/archive/main.tar.gz' into the Git cache...
warning: creating lock file "/tmp/hoowowo/clan/flake.lock":
• Added input 'clan-core':
    '1d8ac7b1b5.tar.gz?narHash=sha256-eBxi0ZMwaALfMsP70N0FRMlOSq0qePv%2BjebVBHXlOqk%3D' (2025-07-25)
• Added input 'clan-core/data-mesher':
    '18dfd42bdb.tar.gz?narHash=sha256-jyoEbaXa8/MwVQ%2BPajUdT63y3gYhgD9o7snO/SLaikw%3D' (2025-07-21)
• Added input 'clan-core/data-mesher/flake-parts':
    follows 'clan-core/flake-parts'
• Added input 'clan-core/data-mesher/nixpkgs':
    follows 'clan-core/nixpkgs'
• Added input 'clan-core/data-mesher/treefmt-nix':
    follows 'clan-core/treefmt-nix'
• Added input 'clan-core/disko':
    'github:nix-community/disko/545aba02960caa78a31bd9a8709a0ad4b6320a5c?narHash=sha256-7lrVrE0jSvZHrxEzvnfHFE/Wkk9DDqb%2BmYCodI5uuB8%3D' (2025-07-21)
• Added input 'clan-core/disko/nixpkgs':
    follows 'clan-core/nixpkgs'
• Added input 'clan-core/flake-parts':
    'github:hercules-ci/flake-parts/644e0fc48951a860279da645ba77fe4a6e814c5e?narHash=sha256-TVcTNvOeWWk1DXljFxVRp%2BE0tzG1LhrVjOGGoMHuXio%3D' (2025-07-21)
• Added input 'clan-core/flake-parts/nixpkgs-lib':
    follows 'clan-core/nixpkgs'
• Added input 'clan-core/nix-darwin':
    'github:nix-darwin/nix-darwin/e04a388232d9a6ba56967ce5b53a8a6f713cdfcf?narHash=sha256-HsJM3XLa43WpG%2B665aGEh8iS8AfEwOIQWk3Mke3e7nk%3D' (2025-06-30)
• Added input 'clan-core/nix-darwin/nixpkgs':
    follows 'clan-core/nixpkgs'
• Added input 'clan-core/nix-select':
    '69d8bf5961.tar.gz?narHash=sha256-IVaoOGDIvAa/8I0sdiiZuKptDldrkDWUNf/%2BezIRhyc%3D' (2025-04-18)
• Added input 'clan-core/nixos-facter-modules':
    'github:nix-community/nixos-facter-modules/14df13c84552a7d1f33c1cd18336128fbc43f920?narHash=sha256-uP9Xxw5XcFwjX9lNoYRpybOnIIe1BHfZu5vJnnPg3Jc%3D' (2025-06-20)
• Added input 'clan-core/nixpkgs':
    'https://releases.nixos.org/nixpkgs/nixpkgs-25.11pre827262.be9e214982e2/nixexprs.tar.xz?narHash=sha256-lUi%2BsPH7Kuh9uP3PyfgbENcJGReUM8Ffk9GxGBFbSN8%3D' (1980-01-01)
• Added input 'clan-core/sops-nix':
    'github:Mic92/sops-nix/2c8def626f54708a9c38a5861866660395bb3461?narHash=sha256-GllP7cmQu7zLZTs9z0J2gIL42IZHa9CBEXwBY9szT0U%3D' (2025-07-15)
• Added input 'clan-core/sops-nix/nixpkgs':
    follows 'clan-core/nixpkgs'
• Added input 'clan-core/systems':
    'github:nix-systems/default/da67096a3b9bf56a91d16901293e51ba5b49a27e?narHash=sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768%3D' (2023-04-09)
• Added input 'clan-core/treefmt-nix':
    'github:numtide/treefmt-nix/421b56313c65a0815a52b424777f55acf0b56ddf?narHash=sha256-tzbhc4XttkyEhswByk5R38l%2BztN9UDbnj0cTcP6Hp9A%3D' (2025-07-20)
• Added input 'clan-core/treefmt-nix/nixpkgs':
    follows 'clan-core/nixpkgs'
• Added input 'flake-parts':
    'github:hercules-ci/flake-parts/644e0fc48951a860279da645ba77fe4a6e814c5e?narHash=sha256-TVcTNvOeWWk1DXljFxVRp%2BE0tzG1LhrVjOGGoMHuXio%3D' (2025-07-21)
• Added input 'flake-parts/nixpkgs-lib':
    follows 'clan-core/nixpkgs'
• Added input 'nixpkgs':
    follows 'clan-core/nixpkgs'
```

Those are 48 lines that seldom carry actual useful information.
This can be shown on running `clan flakes create` with the `--debug`
flag.

Closes: #4496
2025-07-26 14:11:56 +02:00
Kenji Berthold
1d8ac7b1b5 Merge pull request 'pkgs/cli/lib: Allow clan templates list to function outside a clan' (#4490) from kenji/ke-templates-list-without-clan into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4490
Reviewed-by: hsjobeki <hsjobeki@gmail.com>
2025-07-25 14:32:15 +00:00
a-kenji
5b5f1975c5 pkgs/cli/lib: Allow clan templates list to function outside a clan
Allow `clan templates list` to function outside a clan.
Currently when bootstrapping a clan and trying to list the templates
it fails as follows:

```
Traceback (most recent call last):
  File "/nix/store/pkrsr8zr90bps1fwrl8n74zbb9g038b8-clan-cli/bin/.clan-wrapped", line 9, in <module>
    sys.exit(main())
             ~~~~^^
  File "/nix/store/pkrsr8zr90bps1fwrl8n74zbb9g038b8-clan-cli/lib/python3.13/site-packages/clan_cli/cli.py", line 516, in main
    args.func(args)
    ~~~~~~~~~^^^^^^
  File "/nix/store/pkrsr8zr90bps1fwrl8n74zbb9g038b8-clan-cli/lib/python3.13/site-packages/clan_cli/templates/list.py", line 11, in list_command
    templates = list_templates(args.flake)
  File "/nix/store/pkrsr8zr90bps1fwrl8n74zbb9g038b8-clan-cli/lib/python3.13/site-packages/clan_lib/templates/__init__.py", line 20, in list_templates
    custom_templates = flake.select("clanInternals.inventoryClass.templatesPerSource")
                       ^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'select'
```

With the change we get the following output:
```
Available 'clan' templates
├── <builtin>
│   ├── default: Initialize a new clan flake
│   ├── flake-parts: Flake-parts
│   └── minimal: for clans managed via (G)UI
Available 'disko' templates
├── <builtin>
│   └── single-disk: A simple ext4 disk with a single partition
Available 'machine' templates
├── <builtin>
│   ├── flash-installer: Initialize a new flash-installer machine
│   └── new-machine: Initialize a new machine
```

Allowing to check for available templates without needing to have a
clan, which improves the bootstrapping experience.
2025-07-25 16:14:43 +02:00
lassulus
1a5b77d47a refactor: generalize Tor support to SOCKS5 proxy in network module
- Replace Tor-specific implementation with generic SOCKS5 proxy support
- Change `tor_socks` boolean to `socks_port` and `socks_wrapper` parameters
- Move Tor functionality to clan_lib.network.tor submodule
- Add connection context managers to NetworkTechnologyBase
- Improve network abstraction with proper remote() and connection() methods
- Update all callers to use new SOCKS5 proxy interface
- Fix network ping command to properly handle connection contexts

This allows for more flexible proxy configurations beyond just Tor,
while maintaining backward compatibility for Tor usage.
2025-07-24 22:26:44 +02:00
lassulus
9e85c64139 clan-cli flake: show cache file location 2025-07-24 22:24:34 +02:00
Luis Hebendanz
9668c318dc Merge pull request 'fix flake select logging' (#4483) from Qubasa/clan-core:fix_flake_select_logging into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4483
2025-07-24 11:44:21 +00:00