Commit Graph

1527 Commits

Author SHA1 Message Date
Jörg Thalheim
949536bb2b rework cache to use json instead of pickle
Pickle can silently break if migrate our data layout and also introduces
unwanted behaviour such as code injection that we want to avoid.
2025-04-15 07:00:00 +00:00
Jörg Thalheim
833798f650 test_vars_deployment: skip on macOS for now
we can not run vms on macOS at the moment.
2025-04-14 15:47:55 +00:00
Jörg Thalheim
4dc1e2cb3f don't error on macOS if sandbox for vars is missing 2025-04-14 14:11:51 +02:00
Mic92
dc6cf1b673 Merge pull request 'cache nix_config' (#3306) from nix-config into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/3306
2025-04-14 11:23:33 +00:00
Jörg Thalheim
268be9e1f3 cache nix_config
We often call this several times during execution.
At the moment we only use system at the moment, which is unlikely to
change.
2025-04-14 13:13:49 +02:00
Luis Hebendanz
8028876120 Merge pull request 'clan-cli: Don't eval 'modules' while getting templates. 'modules' can be not JSON serializable' (#3280) from Qubasa/clan-core:fix_templates into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/3280
2025-04-14 10:38:07 +00:00
Luis Hebendanz
0c22b31a37 Merge pull request 'chore(clan/cli): remove --reflink=auto' (#3291) from visualphoenix/clan-core:clan_cli_osx_cp_fix into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/3291
2025-04-14 10:24:35 +00:00
Michael Hoang
866dafab9d machines: minor refactor of update 2025-04-13 18:14:17 +02:00
Raymond Barbiero
d16e540671 chore(clan/cli): remove --reflink=auto 2025-04-12 19:41:43 -04:00
Qubasa
b6753211e9 clan-cli: Fix failing tests. Add a big FIXME for clan.select 2025-04-11 23:36:02 +02:00
Qubasa
3bd766e0a0 clan-cli: Fix single file upload in upload.py, add test for edge case 2025-04-11 19:08:13 +02:00
Johannes Kirschbauer
d5a32cc453 chore(clan/vars): make no-sandboxing message print a more explizit re-run command 2025-04-10 22:39:27 +02:00
Qubasa
5a266ecdde clan-cli: Fixup more failing tests because of quoting issues 2025-04-10 14:33:54 +02:00
Qubasa
3fbc661ed9 clan-cli: Change shellcheck to shellcheck-minimal 2025-04-10 01:54:00 +02:00
Qubasa
6b28e4e1d7 clan-cli: Add shellcheck as a runtimeDependencie 2025-04-09 19:44:00 +02:00
RTUnreal
37a627f8bc pkgs/clan-cli: fix test which are using vars shellcheck 2025-04-09 19:44:00 +02:00
Johannes Kirschbauer
61d1cf32ab chore(clan/pytest): discover all test files in the tree 2025-04-09 18:54:57 +02:00
Johannes Kirschbauer
872aa8e5b6 fix(clan/machines): move machineClass attribute selector and add a unit test for class detection 2025-04-09 18:29:43 +02:00
Johannes Kirschbauer
cd11835ec4 Reapply "Merge pull request 'chore(buildClan): move machineClass option into inventory.machines submodule' (#3259) from hsjobeki/clan-core:buildclan-cleanup into main"
This reverts commit 6154b4cb3a.
2025-04-09 18:16:35 +02:00
hsjobeki
af03224d40 Merge pull request 'test(clan/inventory): test deserialization of different inventories' (#3258) from hsjobeki/clan-core:inventory-serde into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/3258
2025-04-09 16:07:58 +00:00
Johannes Kirschbauer
bfd2968509 test(clan/inventory): test deserialization of different inventories 2025-04-09 18:00:00 +02:00
hsjobeki
963248ccff Merge pull request 'treewide(clanLib): rename all occurences of {clan-core,self}.lib to 'clanLib'' (#3263) from hsjobeki/clan-core:lib-cleanup into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/3263
2025-04-09 15:10:42 +00:00
Jörg Thalheim
6154b4cb3a Revert "Merge pull request 'chore(buildClan): move machineClass option into inventory.machines submodule' (#3259) from hsjobeki/clan-core:buildclan-cleanup into main"
This reverts commit 325a794d12, reversing
changes made to afbae04aa6.
2025-04-09 16:08:39 +02:00
Mic92
6f2909bf87 Merge pull request 'feat(clan-cli): support multiple keys for a user' (#3230) from feat/vars-multiple-user-secrets into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/3230
2025-04-09 13:05:24 +00:00
Johannes Kirschbauer
8f6a688336 fix(python/inventory): add machineClass field 2025-04-09 12:50:43 +00:00
Brian McGee
aa4fe27e51 feat(clan-cli): support multiple keys for a user 2025-04-09 09:58:58 +00:00
Mic92
b8e33babec Merge pull request 'installation: speed up by installing to localhost' (#3255) from flakiness into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/3255
2025-04-09 09:58:43 +00:00
Jörg Thalheim
326633d1b8 hardware: only include sshpass if we are actually using it. 2025-04-09 09:52:56 +00:00
Johannes Kirschbauer
e37f16f47a treewide(clanLib): rename all occurences of {clan-core,self}.lib to 'clanLib' 2025-04-09 11:32:46 +02:00
DavHau
ba1ad5bd43 sops: prioritize SOPS_AGE_KEY_FILE over local key
... instead of loading both keys and raise an error

This is important for testing when one wants to override SOPS_AGE_KEY_FILE

New prio: `SOPS_AGE_KEY` > `SOPS_AGE_KEY_FILE` > `~/.config/sops/age/keys.txt`
2025-04-09 08:45:34 +00:00
DavHau
82b6a52c7c vars: add feature --no-sandbox
Raise warning if sandbox cannot be used -> request user to run with --no-sandbox
2025-04-09 15:02:20 +07:00
Johannes Kirschbauer
0ced167912 fix(clan_app/test): fixup after rebase 2025-04-08 21:38:55 +02:00
Johannes Kirschbauer
e9ce7376f2 fix(clan_cli/tests): fix some imports after rebase 2025-04-08 20:26:38 +02:00
Johannes Kirschbauer
4da6cf594d Tests: exclude test folder itself from autmatic dataclass checks 2025-04-08 20:12:03 +02:00
a-kenji
fae630842d pkgs/cli: Move the test folder inside the python module
Move the `tests` folder to `clan_cli/tests`.

As we now want part of our tests to live next to the functions that are
tested - tests that are not in the `/tests` module also need access to
the configured test fixtures that are exposed by the `pytest_plugins`
declaration.

The following folder structure doesn't support this model:

```
├── clan_cli
│   ├── api
│   │    └── api_init_test.py
├── tests/
│   ├── conftest.py
│   └── ...
```

Here `api_init_test.py` even when importing the test functions will not
have the fixtures configured.

There is a way to configure python to import the fixtures from another
[`project/module`](https://docs.pytest.org/en/stable/how-to/fixtures.html#using-fixtures-from-other-projects), but this seems to *generally* be discouraged.

So moving the `conftest.py` to the toplevel and the `/tests` folder into
the toplevel seems to be a sensible choice choice.
2025-04-08 20:12:01 +02:00
Jörg Thalheim
be37194b9a sops: fix compatibility with new format
The new format can have null values now.
2025-04-08 17:44:23 +02:00
Michael Hoang
83a2de0f47 cli: don't update macOS machines 2025-04-08 08:53:29 +02:00
Michael Hoang
b5a57a72a0 build-clan: support constructing darwinConfigurations 2025-04-08 08:53:29 +02:00
DavHau
d7c3039666 add CLAN_NO_COMMIT env var to disable committing
This is required when calling vars generators for generating vars for
tests
2025-04-07 18:59:19 +07:00
Jörg Thalheim
721f61eaed only compute final_script and validation hashes once 2025-04-04 18:31:12 +02:00
Jörg Thalheim
340bf4752f ssh/upload: fix shell quoting 2025-04-04 15:10:09 +00:00
Michael Hoang
7b456be472 cli: managed by clan -> managed by Clan 2025-04-03 00:25:51 +09:00
Michael Hoang
c7ba642a38 cli: machines update [MACHINE] -> machines update [MACHINES] 2025-04-03 00:25:51 +09:00
Qubasa
fb1cea261e clan-cli: Fix deployment as non root edgecase 2025-03-31 20:25:05 +02:00
Johannes Kirschbauer
ae8e7edac0 feat(clanLib): expose clanInternals.clanLib
This is usefully for:
- As api via python
- for testing clanLib downstream
2025-03-30 17:07:11 +02:00
Jonathan Thiessen
89379f103a Make Generator's validation dynamic
* Switch `Generator`'s `validation` from a regular property to
  an `@property` annotated method backed by `Machine`'s `eval_nix()`.
* Ensure that `Machine`'s flake cache is flushed after each
  effectful generator execution (rather than only after all
  generators have been executed).
2025-03-30 04:33:30 +00:00
Jonathan Thiessen
3c0c2ce9d6 Fix cached None support in FlakeCacheEntry
Previously, you could cache None values; however,
insertion wasn't idempotent/identical reinsertion
would lead to errors due to missing None checks.
2025-03-30 04:33:30 +00:00
Jonathan Thiessen
d4fa480262 Add overlapping (consistent) flake cache insert test
* Additionally, update `insert`'s input type hint to support None values
  (as they are already selectable and (one shot) insertable).
  This is necessary to appease the linter wrt the added test.
2025-03-30 04:33:30 +00:00
Luis Hebendanz
3b2ed4e6ab Merge pull request 'clan-cli: Fix deployment as non root with a buildHost set' (#3132) from Qubasa/clan-core:fix_target_host_as_non_root into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/3132
2025-03-29 16:56:49 +00:00
Johannes Kirschbauer
12b88cd19b feat(classgen): convert only certain attributes 2025-03-29 14:22:34 +01:00