Commit Graph

74 Commits

Author SHA1 Message Date
Johannes Kirschbauer
906126f91f Vars/tests: secrets dont raise backend error depending on the previous backend 2025-01-22 09:02:55 +00:00
DavHau
bbe37a998f vars: rename createFile -> persist 2025-01-11 04:19:46 +00:00
Johannes Kirschbauer
c6fe4f2625 API/vars: use string based interfaces to get and set vars to avoid state mutations 2025-01-10 12:06:01 +00:00
Johannes Kirschbauer
6b209f1008 API/vars: simplify get/set prompts 2025-01-07 11:09:58 +01:00
Jörg Thalheim
7466445653 updating groups/machines/users keys now also update vars secrets 2024-12-17 19:24:30 +01:00
Jörg Thalheim
9d9e5bdd26 vars/tests: merge test_build_scripts_for_correct_system into test_generate_secret_for_multiple_machines 2024-12-17 17:39:27 +01:00
Jörg Thalheim
1255c3108e merge three prompt type tests into one 2024-12-17 17:39:19 +01:00
Jörg Thalheim
5ebbb3ee98 vars: remove duplicate test 2024-12-17 17:21:10 +01:00
Jörg Thalheim
ab1626d0eb var: merge migration tests into one 2024-12-17 17:21:10 +01:00
Jörg Thalheim
0ab3ec5e16 vars: delete duplicate shared flag test 2024-12-17 17:21:01 +01:00
Jörg Thalheim
8cc41f543a clan_cli: merge four var tests into one 2024-12-17 17:21:01 +01:00
DavHau
7b3b1018aa vars: commit files for clan vars set
fixes #2601
2024-12-17 15:56:08 +07:00
DavHau
5d47622a78 build-clan: fix bug where vars generator scripts are built for target system instead of local system 2024-12-16 19:20:17 +07:00
lassulus
0ce51a4e5c cli test_vars: make output check less sensible to nix output 2024-12-14 13:42:31 +01:00
Jörg Thalheim
7d7e0c0714 fix clan vars for public vars 2024-12-11 16:39:54 +01:00
DavHau
98ec56b9ee vars: add test to ensure shared secrets access after --regenerate
Ensure that a shared secret can still be accessed by all machines that share it, after a 'clan vars generate --regenerate' has been issued
2024-12-11 15:44:30 +07:00
DavHau
d925f4330d vars: add test for shared secret not being regenerated
see https://git.clan.lol/clan/clan-core/issues/2393
2024-12-11 15:14:44 +07:00
lassulus
3337d7bce6 vars: move ensure_consistent_state into health_check, move into store classes 2024-12-10 11:54:52 +00:00
Jörg Thalheim
d5eb60d904 test_vars: set hostPlatform 2024-12-04 17:25:20 +01:00
DavHau
37ad4eaad5 vars: print() -> log.info() 2024-12-04 14:08:50 +07:00
DavHau
f5226ec7ff vars: Improve logging for migration 2024-12-04 12:42:03 +07:00
DavHau
66ce80c096 vars: rename: invalidation -> validation 2024-11-29 17:23:31 +07:00
DavHau
61576649ff vars: make all python tests work in nix sandbox
- generate a flake.lock file for each template by copying the clan-core flake.lock and modifying it

- call nix build with --store for tests inside the sandbox
2024-11-27 14:32:02 +07:00
Jörg Thalheim
45dfbf54db vars: make interface more type-safe 2024-11-26 17:08:26 +01:00
Jörg Thalheim
8eb37903e0 test_vars: mock ask function instead of sys.stdin 2024-11-26 11:56:38 +00:00
DavHau
173436632d vars: fix migration - secrets end up in public store 2024-11-26 17:02:11 +07:00
DavHau
3f62e143ec vars: implement invalidation mechanism
This adds options `invalidationData` to generators.

`invalidationData` can be used by an author of a generator to signal if a re-generation is required after updating the logic.

Whenever a generator with invalidation data is executed, a hash of that data is stored by the respective public and/or secret backends.

The stored hashes will be checked on future deployments, and a re-generation is triggered whenever a hash doesn't match what's defined in nix.
2024-11-20 16:27:22 +07:00
DavHau
64a6f380a6 vars: add test for 'clan vars get' 2024-11-19 13:32:27 +07:00
DavHau
6a373f92d1 tests: improve performance, keep flake.lock between tests
- deprecate fixture generate_flake in favor of class ClanFlake
- allow creating flake templates that can be copied, including their state, like git and lock files, in order to avoid expensive re-initialization
2024-11-18 17:01:58 +07:00
DavHau
54b8f5904e vars: allow re-encrypting secrets when recipient keys were added.
When the users of a secret change, when for example a new admin user is added, an error will be thrown when generating vars, prompting the user to pass --fix to re-encrypt the secrets
2024-11-13 18:49:30 +07:00
DavHau
7208c6dc82 vars: fix shared dependency was not resolved correctly 2024-10-23 20:43:33 +07:00
DavHau
d9597dab05 vars,facts: add sops.defaultGroups to new machines
Prior to this, when initializing new machines, only the current user had access to it's key which prevents other admin users to deploy this machine later
2024-10-10 19:31:03 +07:00
Jörg Thalheim
1e0f00e6d2 tests: set git author name + email via environment variables 2024-10-09 12:07:58 +02:00
Jörg Thalheim
23c23390a8 re-use gpg key across tests 2024-10-09 11:56:41 +02:00
Jörg Thalheim
67886dc972 fix missing hostPlatform in test_vars 2024-10-09 11:56:36 +02:00
DavHau
a257769abd vars/keygen: fix + cleanup tests
- generate keys in ./sops instead of ./sops/vars for now
- don't initialize all flakes with sops keys, only generate when needed
- use the new 'clan vars keygen' in tests
2024-10-04 17:25:17 +02:00
DavHau
3357a0082e vars: add 'clan vars keygen' 2024-10-04 16:13:49 +02:00
Jörg Thalheim
16fefb27bb add temp_dir fixture 2024-10-01 20:54:19 +02:00
DavHau
0324f4d4b8 vars/sops: improve shared secrets, switching backend
When a second machine checks for a shared secret, now the exists() call returns negative and only when updating the secrets for that machine, the machine is added to the sops receivers.

Also throw proper errors when the user switches backends without cleaning the files first.
2024-09-20 15:06:54 +02:00
DavHau
501ade7de7 vars: implement migration
Migrating generated files from the facts subsystem to the vars subsystem is now possible.

HowTo:
1. declare `clan.core.vars.generators.<generator>.migrateFact = my_service` where `my_service` refers to a service from `clan.core.facts.services`
2. run `clan vers generate your_machine` or `clan machines update your_machine`

Vars will only be migrated for a generator if:
1. The facts service specified via `migrateFact` does exist
2. None of the vars to generate exist yet
3. All public var names exist in the public facts store
4. All secret var names exist in the secret fact store

If the migration is deemed possible, the generator script will not be executed. Instead the files from the public or secret facts store are read and stored into the corresponding vars store
2024-09-19 17:57:03 +02:00
DavHau
89d39186ee vars/generate: improve output when vars are updated
fixes #2076

- print old and new value if possible
- also inform the user if something hasn't changed
2024-09-18 16:57:01 +02:00
DavHau
1c56ef5725 vars: allow setting a default for values 2024-09-17 21:11:38 +02:00
DavHau
cf47c1d51a vars: generate proper commit messages
fixes #2126
2024-09-17 20:22:18 +02:00
DavHau
067d287398 vars/list: display '<not set>' also for secrets 2024-09-12 17:24:29 +02:00
a-kenji
4d2ad709ad vars: use correct paths for value accesses
Use correct paths for value accesses of vars under:

- `per-machine`
- `shared`
2024-09-11 19:02:43 +02:00
DavHau
6fafafa064 vars/password-store: use standard paths; fix share flag 2024-09-09 21:20:03 +02:00
DavHau
b3dd36bdb6 vars: improve generator pipeline
Compute the whole closure of to-be-executed generators upfront before executing anything

Properly compute closures for the 4 different scenarios:

1. full_closure: run all generators for a selected machine in topological order
2. all_missing_closure: run just the missing generators including their dependents
3. requested_closure: run only a selected list of generators including their missing dependencies and their dependents
4. minimal_closure: Run just enough to ensure that the list of selected generators are in a consistent state. Don't execute anything if nothing is missing.
2024-09-06 21:32:17 +02:00
Qubasa
2b5cdd66f9 clan-cli: Make run_vm_in_thread fail fast. Fix generate_flake to support machine_id 2024-09-06 18:53:02 +02:00
DavHau
f161c7ca58 vars: improve generator execution pipeline
- ensure all dependents are re-generated as well
- refactor: separate out computation of generator update closure
2024-09-06 14:30:49 +02:00
DavHau
1254b80a52 vars: add api endpoint set_prompts 2024-09-04 18:06:11 +02:00