Commit Graph

82 Commits

Author SHA1 Message Date
Jörg Thalheim
b5d132b193 do not instantiate vars_generators several times
We should really not have any property that has side effects i.e.
spawning processes.
2025-05-14 19:53:36 +02:00
Jeremy Fleischman
f3c2e8f33f Add /bin/sh to bubblewrap sandbox
I ran into this error when trying to run `opendkim-genkey` in a vars
generator:

```console
=========================================================================== Command ===========================================================================
nix \
    --extra-experimental-features 'nix-command flakes' \
    shell \
    --inputs-from /nix/store/9r3ddw80dz4qzci9pj57ppbh6gy2pgv9-clan-cli/lib/python3.12/site-packages/clan_cli/nixpkgs \
    'nixpkgs#bash' \
    'nixpkgs#bubblewrap' \
    -c bwrap \
    --unshare-all --tmpfs \
    / \
    --ro-bind /nix/store \
    /nix/store \
    --dev /dev \
    --bind /tmp/nix-shell.ClOjgJ/vars-kh4qrnas \
    /tmp/nix-shell.ClOjgJ/vars-kh4qrnas \
    --chdir / \
    --bind /proc \
    /proc \
    --uid 1000 \
    --gid 1000 \
    -- bash \
    -c /nix/store/p0089w4y1w3h535g7ipv4jl4r6mb2hs2-generator-dkim-playground.jflei.com.mail

=========================================================================== Stderr ============================================================================
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = (unset),
	LC_ALL = (unset),
	LC_CTYPE = (unset),
	LC_NUMERIC = (unset),
	LC_COLLATE = (unset),
	LC_TIME = (unset),
	LC_MESSAGES = (unset),
	LC_MONETARY = (unset),
	LC_ADDRESS = (unset),
	LC_IDENTIFICATION = (unset),
	LC_MEASUREMENT = (unset),
	LC_PAPER = (unset),
	LC_TELEPHONE = (unset),
	LC_NAME = (unset),
	LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Can't exec "/bin/sh": No such file or directory at /nix/store/nfawbww80p1hgpymfgq1vq8wqlak75yh-opendkim-2.11.0-Beta2/sbin/.opendkim-genkey-wrapped line 139.
.opendkim-genkey-wrapped: openssl died with signal %d
127
Return Code: 1

1 hosts failed with an error. Check the logs above
```

As we allow `/bin/sh` in the nix build sandbox, I assume we're OK
allowing it here as well?
2025-05-09 18:33:08 -07:00
DavHau
18e75c9954 GUI: initialize support for vars prompts
... for now only when updating a machine (not when installing)

Whenever the user clicks on the update button in the machine view, and only if user input is needed for some missing vars, the user will be forwarded to a vars page.
2025-05-07 18:06:35 +07:00
Qubasa
073304ec6b clan-cli: fix bubblewrap not finding bash when IN_NIX_SANDBOX=1 if prev environment doesn't have it in PATH 2025-05-05 22:19:17 +02:00
DavHau
23d19f1a52 vars: improve API of generate_vars_for_machine
receive list of generator names as an argument instead of generator objects
2025-05-05 15:55:04 +07:00
Mic92
655c7e4eed Merge pull request 'Avoid a few cases of chmod-after-creation' (#3438) from tangential/clan-core:it-s_a_race into main
Reviewed-on: https://git.clan.lol/clan/clan-core/pulls/3438
2025-05-04 07:08:43 +00:00
DavHau
71cdbc989c GUI/vars: add endpoints for getting prompts and generating vars 2025-05-03 14:44:51 +07:00
DavHau
6e9382b942 refactor(vars): move migration logic to extra file 2025-05-03 07:33:11 +00:00
Jonathan Thiessen
9f745ff637 Avoid a few cases of chmod-after-creation 2025-04-28 17:11:21 -07:00
Johannes Kirschbauer
dc284e1c40 vars: move overeager cache invalidation after one generator closure is regenrated.
Invalidation doesn't need to be done after each generator is executed.
We cannot interpolate values from other generators into another
generator. The generators are executed in order. The finalScript of each
generator stays constant.
After the complete closure is generated the caller of generate may
decide to invalidate the flake cache
2025-04-22 16:42:21 +02:00
Jörg Thalheim
723d72255c Reapply "remove nix_shell_legacy"
This reverts commit c5001f19fc.
2025-04-21 13:23:50 +02:00
Johannes Kirschbauer
c5001f19fc Revert "remove nix_shell_legacy"
This reverts commit f3512b853a.
2025-04-18 14:49:54 +02:00
Jörg Thalheim
f3512b853a remove nix_shell_legacy 2025-04-16 21:03:58 +02:00
Jörg Thalheim
837789010e rename nix_shell_legacy to nix_shell and run_cmd to nix_shell
Than it's more obvious that we need to migrate.
2025-04-16 18:27:01 +00:00
lassulus
da92c19367 clan_cli vars generate: prefetch all validationHashes for faster eval 2025-04-14 14:28:59 +02:00
Jörg Thalheim
4dc1e2cb3f don't error on macOS if sandbox for vars is missing 2025-04-14 14:11:51 +02:00
Johannes Kirschbauer
d5a32cc453 chore(clan/vars): make no-sandboxing message print a more explizit re-run command 2025-04-10 22:39:27 +02:00
DavHau
82b6a52c7c vars: add feature --no-sandbox
Raise warning if sandbox cannot be used -> request user to run with --no-sandbox
2025-04-09 15:02:20 +07:00
Jörg Thalheim
721f61eaed only compute final_script and validation hashes once 2025-04-04 18:31:12 +02:00
Jonathan Thiessen
89379f103a Make Generator's validation dynamic
* Switch `Generator`'s `validation` from a regular property to
  an `@property` annotated method backed by `Machine`'s `eval_nix()`.
* Ensure that `Machine`'s flake cache is flushed after each
  effectful generator execution (rather than only after all
  generators have been executed).
2025-03-30 04:33:30 +00:00
Jörg Thalheim
43035b85a5 always resolve symlinks for TemporaryDirectory
On macOS mktemp returns a temporary directory in a symlink.
Nix has a bug where it won't accept path:// located in a symlink.
This avoid this issue by always resolving symlinks as returned by
TemporaryDirectory.
2025-03-19 16:47:18 +01:00
DavHau
db2e2e974c vars+facts: use bwrap only if supported 2025-03-09 13:52:15 +07:00
a-kenji
fa54c0f1b5 Fix various typos 2025-01-22 13:19:28 +01:00
Johannes Kirschbauer
06869a4d27 API/vars: use string based interfaces to get and set vars to avoid state mutations 2025-01-10 12:06:01 +00:00
Jörg Thalheim
5e43571140 vars: commit validation hashes 2024-12-25 21:02:52 +01:00
Jörg Thalheim
7b3efcec06 vars/fact: isolate secret generation better from the system 2024-12-25 19:21:51 +00:00
Michael Hoang
67b9357ce4 vars: fix running generators on macOS 2024-12-19 00:16:22 +11:00
lassulus
0cfb43bada cli vars generate: quote generate name 2024-12-15 13:23:11 +01:00
lassulus
b3f87d1f40 cli vars generate: mount test_store into bwrap 2024-12-14 17:05:27 +01:00
lassulus
9cc3bdbc9f vars: eval finalScript lazy 2024-12-14 13:38:51 +01:00
Qubasa
b9091beff9 clan-cli: Replace log.info to machine.info if applicable 2024-12-12 15:36:17 +01:00
lassulus
998ff92b51 vars: remove intermediate classes 2024-12-10 14:04:31 +01:00
lassulus
9129790e5c vars: move ensure_consistent_state into health_check, move into store classes 2024-12-10 11:54:52 +00:00
Qubasa
65a5789c5b clan-cli: Replace HostGroup and MachineGroup with generic AsyncRuntime class. Propagate cmd prefix over thread local. Close threads on CTRL+C 2024-12-09 18:07:23 +01:00
lassulus
c3f2a1e588 vars migration: raise error on incomplete migration, commit migrated files 2024-12-06 11:25:17 +01:00
clan-bot
2b763152fb Merge pull request 'fix vars migration prompts. add secretsForUsers to vars interface and implement that for pass' (#2551) from lassulus/clan-core:vars-stuff into main 2024-12-04 09:03:24 +00:00
DavHau
a11820b1d6 vars: Improve logging for migration 2024-12-04 12:42:03 +07:00
lassulus
996c5bdda1 cli vars generate: log in global context what is global 2024-12-03 22:29:25 +01:00
lassulus
9f5cd917de vars generate: show prompts only if not migrating 2024-12-03 22:25:16 +01:00
DavHau
8d007867b3 vars/migration: remove useless check 2024-11-29 17:23:31 +07:00
DavHau
5c5a87d416 vars: rename: invalidation -> validation 2024-11-29 17:23:31 +07:00
DavHau
fbbfcc0aa5 vars: generate docs for cli and module 2024-11-29 17:23:31 +07:00
Qubasa
1f98df96e3 clan-cli: cmd.run now has its options extracted to a dataclass 2024-11-28 15:26:37 +01:00
Jörg Thalheim
c9e80f38ca vars: make interface more type-safe 2024-11-26 17:08:26 +01:00
DavHau
1881d7f0a5 vars: fix migration - secrets end up in public store 2024-11-26 17:02:11 +07:00
Qubasa
4e6051acdc docs: Fix nix flake check problem with diskId 2024-11-25 18:39:16 +01:00
Qubasa
979e5e839d clan-cli: Refactor ssh part 2, Refactor custom_logger 2024-11-22 22:08:50 +01:00
DavHau
d4c8b2e4ed vars: implement invalidation mechanism
This adds options `invalidationData` to generators.

`invalidationData` can be used by an author of a generator to signal if a re-generation is required after updating the logic.

Whenever a generator with invalidation data is executed, a hash of that data is stored by the respective public and/or secret backends.

The stored hashes will be checked on future deployments, and a re-generation is triggered whenever a hash doesn't match what's defined in nix.
2024-11-20 16:27:22 +07:00
Jörg Thalheim
68a5d072b2 vars: don't print stack trace if generator fails 2024-11-19 09:46:14 +00:00
Jörg Thalheim
2b270a8951 vars: introduce ensure_machine_has_access method for sops
this should help avoiding overriding existing shared secrets by not
triggering vars regeneration if a machine has no access.

wip
2024-11-19 09:46:14 +00:00