Commit Graph

56 Commits

Author SHA1 Message Date
Michael Hoang
67b9357ce4 vars: fix running generators on macOS 2024-12-19 00:16:22 +11:00
lassulus
0cfb43bada cli vars generate: quote generate name 2024-12-15 13:23:11 +01:00
lassulus
b3f87d1f40 cli vars generate: mount test_store into bwrap 2024-12-14 17:05:27 +01:00
lassulus
9cc3bdbc9f vars: eval finalScript lazy 2024-12-14 13:38:51 +01:00
Qubasa
b9091beff9 clan-cli: Replace log.info to machine.info if applicable 2024-12-12 15:36:17 +01:00
lassulus
998ff92b51 vars: remove intermediate classes 2024-12-10 14:04:31 +01:00
lassulus
9129790e5c vars: move ensure_consistent_state into health_check, move into store classes 2024-12-10 11:54:52 +00:00
Qubasa
65a5789c5b clan-cli: Replace HostGroup and MachineGroup with generic AsyncRuntime class. Propagate cmd prefix over thread local. Close threads on CTRL+C 2024-12-09 18:07:23 +01:00
lassulus
c3f2a1e588 vars migration: raise error on incomplete migration, commit migrated files 2024-12-06 11:25:17 +01:00
clan-bot
2b763152fb Merge pull request 'fix vars migration prompts. add secretsForUsers to vars interface and implement that for pass' (#2551) from lassulus/clan-core:vars-stuff into main 2024-12-04 09:03:24 +00:00
DavHau
a11820b1d6 vars: Improve logging for migration 2024-12-04 12:42:03 +07:00
lassulus
996c5bdda1 cli vars generate: log in global context what is global 2024-12-03 22:29:25 +01:00
lassulus
9f5cd917de vars generate: show prompts only if not migrating 2024-12-03 22:25:16 +01:00
DavHau
8d007867b3 vars/migration: remove useless check 2024-11-29 17:23:31 +07:00
DavHau
5c5a87d416 vars: rename: invalidation -> validation 2024-11-29 17:23:31 +07:00
DavHau
fbbfcc0aa5 vars: generate docs for cli and module 2024-11-29 17:23:31 +07:00
Qubasa
1f98df96e3 clan-cli: cmd.run now has its options extracted to a dataclass 2024-11-28 15:26:37 +01:00
Jörg Thalheim
c9e80f38ca vars: make interface more type-safe 2024-11-26 17:08:26 +01:00
DavHau
1881d7f0a5 vars: fix migration - secrets end up in public store 2024-11-26 17:02:11 +07:00
Qubasa
4e6051acdc docs: Fix nix flake check problem with diskId 2024-11-25 18:39:16 +01:00
Qubasa
979e5e839d clan-cli: Refactor ssh part 2, Refactor custom_logger 2024-11-22 22:08:50 +01:00
DavHau
d4c8b2e4ed vars: implement invalidation mechanism
This adds options `invalidationData` to generators.

`invalidationData` can be used by an author of a generator to signal if a re-generation is required after updating the logic.

Whenever a generator with invalidation data is executed, a hash of that data is stored by the respective public and/or secret backends.

The stored hashes will be checked on future deployments, and a re-generation is triggered whenever a hash doesn't match what's defined in nix.
2024-11-20 16:27:22 +07:00
Jörg Thalheim
68a5d072b2 vars: don't print stack trace if generator fails 2024-11-19 09:46:14 +00:00
Jörg Thalheim
2b270a8951 vars: introduce ensure_machine_has_access method for sops
this should help avoiding overriding existing shared secrets by not
triggering vars regeneration if a machine has no access.

wip
2024-11-19 09:46:14 +00:00
Jörg Thalheim
4de97616bc vars: introduce ensure_machine_has_access method for sops
this should help avoiding overriding existing shared secrets by not
triggering vars regeneration if a machine has no access.

wip
2024-11-14 15:37:55 +00:00
DavHau
8b94bc71bc vars: allow re-encrypting secrets when recipient keys were added.
When the users of a secret change, when for example a new admin user is added, an error will be thrown when generating vars, prompting the user to pass --fix to re-encrypt the secrets
2024-11-13 18:49:30 +07:00
Jörg Thalheim
eb1daad08d vars: update message if vars are up-to-date 2024-11-08 15:43:10 +01:00
DavHau
33d049915c vars: fix shared dependency was not resolved correctly 2024-10-23 20:43:33 +07:00
DavHau
b379bffeea vars: show full var_id when prompting 2024-10-16 13:13:41 +07:00
Jörg Thalheim
db11e7cb92 {vars,facts}/generate: ensure that args.flake is passed 2024-10-08 17:51:55 +00:00
Jörg Thalheim
f9a42831e4 start to name temporary directories in more places 2024-10-01 18:57:59 +02:00
Jörg Thalheim
a5e18fda08 generate: don't set type=bool in argparse.BooleanOptionalAction 2024-10-01 18:48:54 +02:00
DavHau
0a8b7f38a8 vars: implement migration
Migrating generated files from the facts subsystem to the vars subsystem is now possible.

HowTo:
1. declare `clan.core.vars.generators.<generator>.migrateFact = my_service` where `my_service` refers to a service from `clan.core.facts.services`
2. run `clan vers generate your_machine` or `clan machines update your_machine`

Vars will only be migrated for a generator if:
1. The facts service specified via `migrateFact` does exist
2. None of the vars to generate exist yet
3. All public var names exist in the public facts store
4. All secret var names exist in the secret fact store

If the migration is deemed possible, the generator script will not be executed. Instead the files from the public or secret facts store are read and stored into the corresponding vars store
2024-09-19 17:57:03 +02:00
DavHau
01c833f2bc vars/generate: improve output when vars are updated
fixes #2076

- print old and new value if possible
- also inform the user if something hasn't changed
2024-09-18 16:57:01 +02:00
DavHau
bd79e6aef6 vars: generate proper commit messages
fixes #2126
2024-09-17 20:22:18 +02:00
DavHau
803f43dfd1 vars: improve generator pipeline
Compute the whole closure of to-be-executed generators upfront before executing anything

Properly compute closures for the 4 different scenarios:

1. full_closure: run all generators for a selected machine in topological order
2. all_missing_closure: run just the missing generators including their dependents
3. requested_closure: run only a selected list of generators including their missing dependencies and their dependents
4. minimal_closure: Run just enough to ensure that the list of selected generators are in a consistent state. Don't execute anything if nothing is missing.
2024-09-06 21:32:17 +02:00
DavHau
6bb93d087c vars: refactor - ask prompts before running any generators 2024-09-06 16:08:46 +02:00
DavHau
5cd9960ed4 vars: improve generator execution pipeline
- ensure all dependents are re-generated as well
- refactor: separate out computation of generator update closure
2024-09-06 14:30:49 +02:00
DavHau
a57525d5c6 vars: add api endpoint set_prompts 2024-09-04 18:06:11 +02:00
clan-bot
87d47c7c31 Merge pull request 'apply TRY lint' (#2035) from joerg-ci into main 2024-09-04 13:20:11 +00:00
DavHau
3f23ad4f79 vars: set vars via cli; improve getting vars via cli; 2024-09-04 14:48:42 +02:00
Jörg Thalheim
29909e49d3 apply TRY lint 2024-09-03 18:13:46 +02:00
DavHau
6e16e54968 vars: make simplify vars store abstraction 2024-09-02 18:44:00 +02:00
Jörg Thalheim
e150b37fb8 enable ASYNC, DTZ, YTT and EM lints 2024-09-02 14:07:06 +02:00
Jörg Thalheim
b313f2d066 make all same-module imports relative, the rest absolute
This makes sorting more consitent.
2024-09-02 13:00:19 +02:00
DavHau
64d29cd28c vars: improve check command and add tests 2024-09-01 16:45:09 +02:00
DavHau
62ccd0ed4b vars: introduce deploy=true/false for generated files 2024-09-01 14:32:46 +02:00
DavHau
d1c2f0b622 vars: introduce share flag 2024-08-03 15:26:53 +07:00
DavHau
0acf9178c8 vars: implement prompts 2024-07-22 20:40:12 +07:00
Qubasa
274701b381 clan: Fix treefmt not applying mypy lints on clan_cli and clan_app 2024-07-17 21:50:52 +02:00