Commit Graph

2697 Commits

Author SHA1 Message Date
a-kenji
e167137672 pkgs/cli/lib: Remove obsolete clan creation function 2025-07-30 12:41:42 +02:00
DavHau
b30686269b machines update: fix lacks a signature by a trusted key
Despite using `root` as the ssh user, `ssh-ng` still fails with:

`error: cannot add path '/nix/store/...' because it lacks a signature by a trusted key
`

This does not happen with `ssh` instead of `ssh-ng`
2025-07-30 15:15:57 +07:00
Kenji Berthold
db579e169c Merge pull request 'pkgs/clan/lib: Fix directory functionality' (#4519) from kenji/ke-fix-directory-usage into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4519
2025-07-29 16:01:35 +00:00
a-kenji
31438d6781 pkgs/clan/lib: Fix directory functionality
Fix the directory functionality of clan (clan.directory).
The python API interface was not able to distinguish if the directory
was set to anything other than `self.src`.
Breaking every command that relied on the clan directory, for example:
- `clan machines update`
- `clan machines update-hardware-config`
See more in #2906

This is the first step in fixing all those commands.
Individual command support and implementation will be implemented in
follow ups.
2025-07-29 17:51:12 +02:00
Kenji Berthold
eac21c5176 Merge pull request 'pkgs/clan/lib: Fix documentation of from_ssh_uri in the Remote class' (#4523) from kenji/ke-fix-remote-documentation into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4523
2025-07-29 15:47:47 +00:00
a-kenji
7ef09343ed pkgs/clan/lib: Fix documentation of from_ssh_uri in the Remote class 2025-07-29 17:37:50 +02:00
a-kenji
8c2cee0e44 pkgs/clan/cli: Fix typo in machines update 2025-07-29 17:32:45 +02:00
hsjobeki
b421698f70 Merge pull request 'templates: fix urls for relative file paths' (#4520) from fix-template-urls into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4520
2025-07-29 15:19:20 +00:00
hsjobeki
857b9d0260 Merge pull request 'docs/templates: add more docs for template urls' (#4521) from docs-templates into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4521
2025-07-29 15:11:15 +00:00
Johannes Kirschbauer
2776294de0 templates: url add support for home and abspath 2025-07-29 17:04:15 +02:00
Johannes Kirschbauer
c90b8d7401 templates/cli: add more help 2025-07-29 16:58:19 +02:00
Johannes Kirschbauer
7784df8180 templates: fix urls for relative file paths 2025-07-29 15:01:29 +02:00
Kenji Berthold
5d0ca5aff8 Merge pull request 'pkgs/clan/lib: Fix clan template creation when already in a flake' (#4501) from kenji/ke-clan-flakes-create-existing-flake-fix into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4501
2025-07-29 11:19:04 +00:00
a-kenji
3ef6b2f715 pkgs/clan/cli: Add test for builtin flakeref 2025-07-29 13:07:48 +02:00
Kenji Berthold
58053748b9 Merge pull request 'pkgs/clan/cli: Add clan flake validation to clan vars check' (#4513) from kenji/ke-vars-check-validation into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4513
2025-07-29 11:03:45 +00:00
DavHau
19a8101e98 Merge pull request 'pkgs/cli/create: Show less output by default' (#4499) from kenji/ke-create-show-less-output into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4499
2025-07-29 10:45:16 +00:00
a-kenji
35315d9596 pkgs/clan/cli: Add clan flake validation to clan vars check
This now gives a clearer error than:
```
Traceback (most recent call last):
  File "/nix/store/mznnb8il3njp6jxn5i57d0myjdh6cs0i-clan-cli/bin/.clan-wrapped", line 9, in <module>
    sys.exit(main())
             ~~~~^^
  File "/nix/store/mznnb8il3njp6jxn5i57d0myjdh6cs0i-clan-cli/lib/python3.13/site-packages/clan_cli/cli.py", line 516, in main
    args.func(args)
    ~~~~~~~~~^^^^^^
  File "/nix/store/mznnb8il3njp6jxn5i57d0myjdh6cs0i-clan-cli/lib/python3.13/site-packages/clan_cli/vars/check.py", line 113, in check_command
    ok = check_vars(args.machine, args.flake, generator_name=args.generator)
  File "/nix/store/mznnb8il3njp6jxn5i57d0myjdh6cs0i-clan-cli/lib/python3.13/site-packages/clan_cli/vars/check.py", line 103, in check_vars
    status = vars_status(machine_name, flake, generator_name=generator_name)
  File "/nix/store/mznnb8il3njp6jxn5i57d0myjdh6cs0i-clan-cli/lib/python3.13/site-packages/clan_cli/vars/check.py", line 41, in vars_status
    generators = Generator.generators_from_flake(machine.name, machine.flake)
  File "/nix/store/mznnb8il3njp6jxn5i57d0myjdh6cs0i-clan-cli/lib/python3.13/site-packages/clan_cli/vars/generate.py", line 67, in generators_from_flake
    generators_data = flake.select_machine(
                      ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'select_machine'
```

When not being in a flake.
2025-07-29 10:25:28 +02:00
a-kenji
86ac1c4405 pkgs/cli: Validate clan flake for clan machines list 2025-07-29 10:14:34 +02:00
DavHau
cb89fb97f1 clan machines update: add --fetch-local feature
Motivation: updating a machine fails, if it depends on a private github repo, as the remote will likely not be authenticated.

This adds a new flag `--fetch-local` to `clan machines update` which fetches all flake inputs prior to building, then uploads them to the build-host.

This also adds a new error message, when flake inputs could not fetched, to hint the user to use `--fetch-local`
2025-07-28 17:01:42 +07:00
hsjobeki
6a8d7aa5fd Merge pull request 'api: init get_machine_writeability' (#4504) from cli-fixes into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4504
2025-07-28 08:47:43 +00:00
Luis Hebendanz
63bcfc4809 Merge pull request 'pkgs/cli: Remove uncommented logic from creation test' (#4497) from kenji/ke-remove-uncommented into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4497
2025-07-28 03:48:57 +00:00
Johannes Kirschbauer
e73350f6af test: fix add modules 2025-07-27 12:48:04 +02:00
Johannes Kirschbauer
98a0b9600b api/writability: add docstring 2025-07-27 12:47:37 +02:00
Johannes Kirschbauer
abeb517a22 api/writability: add unit test 2025-07-27 00:03:05 +02:00
Johannes Kirschbauer
fbdbcfa6d5 InventoryStore: factor write into _write for actual disk interaction 2025-07-27 00:02:47 +02:00
Johannes Kirschbauer
303af9af6b api: init get_machine_writeability 2025-07-27 00:01:51 +02:00
Johannes Kirschbauer
414e412e7e persist/writeability: expose is writeable key helper 2025-07-27 00:01:36 +02:00
Johannes Kirschbauer
c2e84f11af persist/util: add field helper 2025-07-27 00:01:07 +02:00
Johannes Kirschbauer
bf2eb000d5 api/set_machine: add unit tests 2025-07-26 23:59:51 +02:00
Kenji Berthold
6e904de655 Merge pull request 'pkgs/cli: machines install handle invalid character' (#4488) from kenji/ke-clan-machines-install-prompt into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4488
2025-07-26 13:29:10 +00:00
a-kenji
0a43721a45 pkgs/clan/lib: Fix clan template creation when already in a flake
Fix clan template creation when already in a flake.
Currently we already fail with very clear and descriptive error when
trying to evaluate the template of the flake we are in:
```
Failed to select template 'flake-parts' from flake '/tmp/superclan' (via attribute path: /tmp/superclan#clanInternals.templates.clan."flake-parts")
```

This is undesired behavior.
When we are trying to create a clan with `clan flakes create`.
We can't rely on the fact that the flake we are currently in exports flake templates.

Now we *try* to evaluate the flake we are in upon creation.
If there are no clan templates available, we now will fall back to
builtin templates.

Closes: #4472
2025-07-26 15:21:44 +02:00
a-kenji
51eb7bd0b5 pkgs/cli/create: Show less output by default
This masks the output of `nix flake update`, since it is quite verbose
and takes attention away to usually more interesting and pressing
information.

Example:
```
unpacking 'https://git.clan.lol/clan/clan-core/archive/main.tar.gz' into the Git cache...
warning: creating lock file "/tmp/hoowowo/clan/flake.lock":
• Added input 'clan-core':
    '1d8ac7b1b5.tar.gz?narHash=sha256-eBxi0ZMwaALfMsP70N0FRMlOSq0qePv%2BjebVBHXlOqk%3D' (2025-07-25)
• Added input 'clan-core/data-mesher':
    '18dfd42bdb.tar.gz?narHash=sha256-jyoEbaXa8/MwVQ%2BPajUdT63y3gYhgD9o7snO/SLaikw%3D' (2025-07-21)
• Added input 'clan-core/data-mesher/flake-parts':
    follows 'clan-core/flake-parts'
• Added input 'clan-core/data-mesher/nixpkgs':
    follows 'clan-core/nixpkgs'
• Added input 'clan-core/data-mesher/treefmt-nix':
    follows 'clan-core/treefmt-nix'
• Added input 'clan-core/disko':
    'github:nix-community/disko/545aba02960caa78a31bd9a8709a0ad4b6320a5c?narHash=sha256-7lrVrE0jSvZHrxEzvnfHFE/Wkk9DDqb%2BmYCodI5uuB8%3D' (2025-07-21)
• Added input 'clan-core/disko/nixpkgs':
    follows 'clan-core/nixpkgs'
• Added input 'clan-core/flake-parts':
    'github:hercules-ci/flake-parts/644e0fc48951a860279da645ba77fe4a6e814c5e?narHash=sha256-TVcTNvOeWWk1DXljFxVRp%2BE0tzG1LhrVjOGGoMHuXio%3D' (2025-07-21)
• Added input 'clan-core/flake-parts/nixpkgs-lib':
    follows 'clan-core/nixpkgs'
• Added input 'clan-core/nix-darwin':
    'github:nix-darwin/nix-darwin/e04a388232d9a6ba56967ce5b53a8a6f713cdfcf?narHash=sha256-HsJM3XLa43WpG%2B665aGEh8iS8AfEwOIQWk3Mke3e7nk%3D' (2025-06-30)
• Added input 'clan-core/nix-darwin/nixpkgs':
    follows 'clan-core/nixpkgs'
• Added input 'clan-core/nix-select':
    '69d8bf5961.tar.gz?narHash=sha256-IVaoOGDIvAa/8I0sdiiZuKptDldrkDWUNf/%2BezIRhyc%3D' (2025-04-18)
• Added input 'clan-core/nixos-facter-modules':
    'github:nix-community/nixos-facter-modules/14df13c84552a7d1f33c1cd18336128fbc43f920?narHash=sha256-uP9Xxw5XcFwjX9lNoYRpybOnIIe1BHfZu5vJnnPg3Jc%3D' (2025-06-20)
• Added input 'clan-core/nixpkgs':
    'https://releases.nixos.org/nixpkgs/nixpkgs-25.11pre827262.be9e214982e2/nixexprs.tar.xz?narHash=sha256-lUi%2BsPH7Kuh9uP3PyfgbENcJGReUM8Ffk9GxGBFbSN8%3D' (1980-01-01)
• Added input 'clan-core/sops-nix':
    'github:Mic92/sops-nix/2c8def626f54708a9c38a5861866660395bb3461?narHash=sha256-GllP7cmQu7zLZTs9z0J2gIL42IZHa9CBEXwBY9szT0U%3D' (2025-07-15)
• Added input 'clan-core/sops-nix/nixpkgs':
    follows 'clan-core/nixpkgs'
• Added input 'clan-core/systems':
    'github:nix-systems/default/da67096a3b9bf56a91d16901293e51ba5b49a27e?narHash=sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768%3D' (2023-04-09)
• Added input 'clan-core/treefmt-nix':
    'github:numtide/treefmt-nix/421b56313c65a0815a52b424777f55acf0b56ddf?narHash=sha256-tzbhc4XttkyEhswByk5R38l%2BztN9UDbnj0cTcP6Hp9A%3D' (2025-07-20)
• Added input 'clan-core/treefmt-nix/nixpkgs':
    follows 'clan-core/nixpkgs'
• Added input 'flake-parts':
    'github:hercules-ci/flake-parts/644e0fc48951a860279da645ba77fe4a6e814c5e?narHash=sha256-TVcTNvOeWWk1DXljFxVRp%2BE0tzG1LhrVjOGGoMHuXio%3D' (2025-07-21)
• Added input 'flake-parts/nixpkgs-lib':
    follows 'clan-core/nixpkgs'
• Added input 'nixpkgs':
    follows 'clan-core/nixpkgs'
```

Those are 48 lines that seldom carry actual useful information.
This can be shown on running `clan flakes create` with the `--debug`
flag.

Closes: #4496
2025-07-26 14:11:56 +02:00
Kenji Berthold
1d8ac7b1b5 Merge pull request 'pkgs/cli/lib: Allow clan templates list to function outside a clan' (#4490) from kenji/ke-templates-list-without-clan into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4490
Reviewed-by: hsjobeki <hsjobeki@gmail.com>
2025-07-25 14:32:15 +00:00
a-kenji
5b5f1975c5 pkgs/cli/lib: Allow clan templates list to function outside a clan
Allow `clan templates list` to function outside a clan.
Currently when bootstrapping a clan and trying to list the templates
it fails as follows:

```
Traceback (most recent call last):
  File "/nix/store/pkrsr8zr90bps1fwrl8n74zbb9g038b8-clan-cli/bin/.clan-wrapped", line 9, in <module>
    sys.exit(main())
             ~~~~^^
  File "/nix/store/pkrsr8zr90bps1fwrl8n74zbb9g038b8-clan-cli/lib/python3.13/site-packages/clan_cli/cli.py", line 516, in main
    args.func(args)
    ~~~~~~~~~^^^^^^
  File "/nix/store/pkrsr8zr90bps1fwrl8n74zbb9g038b8-clan-cli/lib/python3.13/site-packages/clan_cli/templates/list.py", line 11, in list_command
    templates = list_templates(args.flake)
  File "/nix/store/pkrsr8zr90bps1fwrl8n74zbb9g038b8-clan-cli/lib/python3.13/site-packages/clan_lib/templates/__init__.py", line 20, in list_templates
    custom_templates = flake.select("clanInternals.inventoryClass.templatesPerSource")
                       ^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'select'
```

With the change we get the following output:
```
Available 'clan' templates
├── <builtin>
│   ├── default: Initialize a new clan flake
│   ├── flake-parts: Flake-parts
│   └── minimal: for clans managed via (G)UI
Available 'disko' templates
├── <builtin>
│   └── single-disk: A simple ext4 disk with a single partition
Available 'machine' templates
├── <builtin>
│   ├── flash-installer: Initialize a new flash-installer machine
│   └── new-machine: Initialize a new machine
```

Allowing to check for available templates without needing to have a
clan, which improves the bootstrapping experience.
2025-07-25 16:14:43 +02:00
a-kenji
326f418c88 pkgs/cli: Remove uncommented logic from creation test 2025-07-25 14:44:20 +02:00
lassulus
1a5b77d47a refactor: generalize Tor support to SOCKS5 proxy in network module
- Replace Tor-specific implementation with generic SOCKS5 proxy support
- Change `tor_socks` boolean to `socks_port` and `socks_wrapper` parameters
- Move Tor functionality to clan_lib.network.tor submodule
- Add connection context managers to NetworkTechnologyBase
- Improve network abstraction with proper remote() and connection() methods
- Update all callers to use new SOCKS5 proxy interface
- Fix network ping command to properly handle connection contexts

This allows for more flexible proxy configurations beyond just Tor,
while maintaining backward compatibility for Tor usage.
2025-07-24 22:26:44 +02:00
lassulus
9e85c64139 clan-cli flake: show cache file location 2025-07-24 22:24:34 +02:00
lassulus
7dd9e6b97c clan-cli vars: show which var we are getting in debug log 2025-07-24 22:24:15 +02:00
a-kenji
6cea3e6c60 pkgs/cli: machines install handle invalid character
Re-request prompt, if invalid character is specified.
None is still treated as no as per CLI hint [y/N].
We now also accept Y/N.

Closes: #4475
2025-07-24 22:00:31 +02:00
Luis Hebendanz
9668c318dc Merge pull request 'fix flake select logging' (#4483) from Qubasa/clan-core:fix_flake_select_logging into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4483
2025-07-24 11:44:21 +00:00
Qubasa
7146c97362 clan_lib: Fix flake.select logging, now we log the first time select queries a path for the first time, it doesn't matter if it is cached or not. 2025-07-24 18:27:40 +07:00
Qubasa
b74aa31b87 clan-lib: Fix missing logging for flake.select execution 2025-07-24 17:29:09 +07:00
Qubasa
94662b722d clan-lib: Remove injected "op_key" argument from all functions and do it over the threadcontext instead. Remove double threading in http server 2025-07-24 14:25:20 +07:00
hsjobeki
1e51439414 Merge pull request 'pytest: add simple clan_flake function' (#4453) from api-fixes into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4453
2025-07-23 10:27:05 +00:00
Johannes Kirschbauer
29c764773f pytest: clan_flake allow usage of plain dicts 2025-07-23 12:15:54 +02:00
Brian McGee
38d62af1ba feat(ui): add sidebar and flesh out app routes 2025-07-23 10:16:00 +01:00
a-kenji
f58a120db1 pkgs/cli: Fix fstring interplolation
Closes: #4458
2025-07-23 09:43:51 +02:00
DavHau
cc69892e3b create clan: better info about existing sop keys
When creating a new clan, the key selection now looks like this:
```
Found existing admin keys on this machine:
1: type: AGE
   pubkey: age1xyz...
   source: /home/grmpf/.config/sops/age/keys.txt
2: type: PGP
   pubkey: abc...
   source: SOPS_PGP_FP
Select keys to use (comma-separated list of numbers, or leave empty to select all):
```

This is achieved by adding a `source` attribute to `SopsKey`.
2025-07-23 13:22:19 +07:00
DavHau
c94330ee9c clan create: fix failure when path was single word
This should better be fixed with types. It should be possible to initialize a flake from a Path, making it very clear that a path `foo` is meant and not a remote flake called `foo`
2025-07-23 12:33:57 +07:00
Jörg Thalheim
377056e80c clan flakes create: initialize keys automatically (#4435)
fixes https://git.clan.lol/clan/clan-core/issues/2665
fixes https://git.clan.lol/clan/clan-core/issues/4407

Co-authored-by: DavHau <d.hauer.it@gmail.com>
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/4435
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
Co-committed-by: Jörg Thalheim <joerg@thalheim.io>
2025-07-23 04:44:55 +00:00