Commit Graph

73 Commits

Author SHA1 Message Date
Johannes Kirschbauer
dc284e1c40 vars: move overeager cache invalidation after one generator closure is regenrated.
Invalidation doesn't need to be done after each generator is executed.
We cannot interpolate values from other generators into another
generator. The generators are executed in order. The finalScript of each
generator stays constant.
After the complete closure is generated the caller of generate may
decide to invalidate the flake cache
2025-04-22 16:42:21 +02:00
Jörg Thalheim
723d72255c Reapply "remove nix_shell_legacy"
This reverts commit c5001f19fc.
2025-04-21 13:23:50 +02:00
Johannes Kirschbauer
c5001f19fc Revert "remove nix_shell_legacy"
This reverts commit f3512b853a.
2025-04-18 14:49:54 +02:00
Jörg Thalheim
f3512b853a remove nix_shell_legacy 2025-04-16 21:03:58 +02:00
Jörg Thalheim
837789010e rename nix_shell_legacy to nix_shell and run_cmd to nix_shell
Than it's more obvious that we need to migrate.
2025-04-16 18:27:01 +00:00
lassulus
da92c19367 clan_cli vars generate: prefetch all validationHashes for faster eval 2025-04-14 14:28:59 +02:00
Jörg Thalheim
4dc1e2cb3f don't error on macOS if sandbox for vars is missing 2025-04-14 14:11:51 +02:00
Johannes Kirschbauer
d5a32cc453 chore(clan/vars): make no-sandboxing message print a more explizit re-run command 2025-04-10 22:39:27 +02:00
DavHau
82b6a52c7c vars: add feature --no-sandbox
Raise warning if sandbox cannot be used -> request user to run with --no-sandbox
2025-04-09 15:02:20 +07:00
Jörg Thalheim
721f61eaed only compute final_script and validation hashes once 2025-04-04 18:31:12 +02:00
Jonathan Thiessen
89379f103a Make Generator's validation dynamic
* Switch `Generator`'s `validation` from a regular property to
  an `@property` annotated method backed by `Machine`'s `eval_nix()`.
* Ensure that `Machine`'s flake cache is flushed after each
  effectful generator execution (rather than only after all
  generators have been executed).
2025-03-30 04:33:30 +00:00
Jörg Thalheim
43035b85a5 always resolve symlinks for TemporaryDirectory
On macOS mktemp returns a temporary directory in a symlink.
Nix has a bug where it won't accept path:// located in a symlink.
This avoid this issue by always resolving symlinks as returned by
TemporaryDirectory.
2025-03-19 16:47:18 +01:00
DavHau
db2e2e974c vars+facts: use bwrap only if supported 2025-03-09 13:52:15 +07:00
a-kenji
fa54c0f1b5 Fix various typos 2025-01-22 13:19:28 +01:00
Johannes Kirschbauer
06869a4d27 API/vars: use string based interfaces to get and set vars to avoid state mutations 2025-01-10 12:06:01 +00:00
Jörg Thalheim
5e43571140 vars: commit validation hashes 2024-12-25 21:02:52 +01:00
Jörg Thalheim
7b3efcec06 vars/fact: isolate secret generation better from the system 2024-12-25 19:21:51 +00:00
Michael Hoang
67b9357ce4 vars: fix running generators on macOS 2024-12-19 00:16:22 +11:00
lassulus
0cfb43bada cli vars generate: quote generate name 2024-12-15 13:23:11 +01:00
lassulus
b3f87d1f40 cli vars generate: mount test_store into bwrap 2024-12-14 17:05:27 +01:00
lassulus
9cc3bdbc9f vars: eval finalScript lazy 2024-12-14 13:38:51 +01:00
Qubasa
b9091beff9 clan-cli: Replace log.info to machine.info if applicable 2024-12-12 15:36:17 +01:00
lassulus
998ff92b51 vars: remove intermediate classes 2024-12-10 14:04:31 +01:00
lassulus
9129790e5c vars: move ensure_consistent_state into health_check, move into store classes 2024-12-10 11:54:52 +00:00
Qubasa
65a5789c5b clan-cli: Replace HostGroup and MachineGroup with generic AsyncRuntime class. Propagate cmd prefix over thread local. Close threads on CTRL+C 2024-12-09 18:07:23 +01:00
lassulus
c3f2a1e588 vars migration: raise error on incomplete migration, commit migrated files 2024-12-06 11:25:17 +01:00
clan-bot
2b763152fb Merge pull request 'fix vars migration prompts. add secretsForUsers to vars interface and implement that for pass' (#2551) from lassulus/clan-core:vars-stuff into main 2024-12-04 09:03:24 +00:00
DavHau
a11820b1d6 vars: Improve logging for migration 2024-12-04 12:42:03 +07:00
lassulus
996c5bdda1 cli vars generate: log in global context what is global 2024-12-03 22:29:25 +01:00
lassulus
9f5cd917de vars generate: show prompts only if not migrating 2024-12-03 22:25:16 +01:00
DavHau
8d007867b3 vars/migration: remove useless check 2024-11-29 17:23:31 +07:00
DavHau
5c5a87d416 vars: rename: invalidation -> validation 2024-11-29 17:23:31 +07:00
DavHau
fbbfcc0aa5 vars: generate docs for cli and module 2024-11-29 17:23:31 +07:00
Qubasa
1f98df96e3 clan-cli: cmd.run now has its options extracted to a dataclass 2024-11-28 15:26:37 +01:00
Jörg Thalheim
c9e80f38ca vars: make interface more type-safe 2024-11-26 17:08:26 +01:00
DavHau
1881d7f0a5 vars: fix migration - secrets end up in public store 2024-11-26 17:02:11 +07:00
Qubasa
4e6051acdc docs: Fix nix flake check problem with diskId 2024-11-25 18:39:16 +01:00
Qubasa
979e5e839d clan-cli: Refactor ssh part 2, Refactor custom_logger 2024-11-22 22:08:50 +01:00
DavHau
d4c8b2e4ed vars: implement invalidation mechanism
This adds options `invalidationData` to generators.

`invalidationData` can be used by an author of a generator to signal if a re-generation is required after updating the logic.

Whenever a generator with invalidation data is executed, a hash of that data is stored by the respective public and/or secret backends.

The stored hashes will be checked on future deployments, and a re-generation is triggered whenever a hash doesn't match what's defined in nix.
2024-11-20 16:27:22 +07:00
Jörg Thalheim
68a5d072b2 vars: don't print stack trace if generator fails 2024-11-19 09:46:14 +00:00
Jörg Thalheim
2b270a8951 vars: introduce ensure_machine_has_access method for sops
this should help avoiding overriding existing shared secrets by not
triggering vars regeneration if a machine has no access.

wip
2024-11-19 09:46:14 +00:00
Jörg Thalheim
4de97616bc vars: introduce ensure_machine_has_access method for sops
this should help avoiding overriding existing shared secrets by not
triggering vars regeneration if a machine has no access.

wip
2024-11-14 15:37:55 +00:00
DavHau
8b94bc71bc vars: allow re-encrypting secrets when recipient keys were added.
When the users of a secret change, when for example a new admin user is added, an error will be thrown when generating vars, prompting the user to pass --fix to re-encrypt the secrets
2024-11-13 18:49:30 +07:00
Jörg Thalheim
eb1daad08d vars: update message if vars are up-to-date 2024-11-08 15:43:10 +01:00
DavHau
33d049915c vars: fix shared dependency was not resolved correctly 2024-10-23 20:43:33 +07:00
DavHau
b379bffeea vars: show full var_id when prompting 2024-10-16 13:13:41 +07:00
Jörg Thalheim
db11e7cb92 {vars,facts}/generate: ensure that args.flake is passed 2024-10-08 17:51:55 +00:00
Jörg Thalheim
f9a42831e4 start to name temporary directories in more places 2024-10-01 18:57:59 +02:00
Jörg Thalheim
a5e18fda08 generate: don't set type=bool in argparse.BooleanOptionalAction 2024-10-01 18:48:54 +02:00
DavHau
0a8b7f38a8 vars: implement migration
Migrating generated files from the facts subsystem to the vars subsystem is now possible.

HowTo:
1. declare `clan.core.vars.generators.<generator>.migrateFact = my_service` where `my_service` refers to a service from `clan.core.facts.services`
2. run `clan vers generate your_machine` or `clan machines update your_machine`

Vars will only be migrated for a generator if:
1. The facts service specified via `migrateFact` does exist
2. None of the vars to generate exist yet
3. All public var names exist in the public facts store
4. All secret var names exist in the secret fact store

If the migration is deemed possible, the generator script will not be executed. Instead the files from the public or secret facts store are read and stored into the corresponding vars store
2024-09-19 17:57:03 +02:00