---
.title = "NixOS on new systems",
.date = @date("2025-10-28"),
.author = "Yadunand Prem",
.layout = "post.shtml",
.draft = false,
.tags = ["k3s", "kubernetes", "nutinfra", "nix", "nixOS"],
---
# Fresh installing NixOS on a new system

So greencloudVPS just had its 12th birthday sale, and with greed, i bought their server bundle... 220 USD/3 years for a 4 core, 12GB ram server with 4 TB storage was a server deal I could not give up on... So with 4TB, I wanted it as my offsite backup server + another node for my kubernetes cluster. And now with 2 public IPs, I could do fun stuff like floating IP for more reliability. 

Another thing I want to do is slowly migrate away from my 1 \$5/month digital ocean server, which mainly runs headscale, to one of these servers instead. 

So the question is, do I really need headscale, or can I just use plain wireguard and with with nix, is it much easier to push my changes to a cluster now with tools like colmena? My [homelab](https://github.com/yadunut/homelab) is using colmena right now, but I'm slowly migrating to [this repo](https://git.yadunut.dev/yadunut/nix) for all my nix needs.

Another thing to look into is completely restructuring my current infra setup, now that I have more than 1 cloud node. 

But I digress, this article is on installing nixOS on this system. 

## The tools
- [nixos anywhere](https://github.com/nix-community/nixos-anywhere)
  NixOS anywhere, as its name suggests, is a tool to install nixOS on any linux system supporting kexec over ssh. 
- [disko](https://github.com/nix-community/disko)
  Nix tool for declarative disk partitioning

## Setup
asd

### Figuring out partitioning with disko
The nut-gc2 server has 2 drives, vda at 60GB, and vdb, at 4TB. I want the vda to be the primary partition, storing the root file system, and VDB just being used for longhorn. 

So lets get that out in disko. I initially used UEFI but i was having issues with it booting up, so I'm switching to BIOS. 
```nix
{
  disko.devices = {
    disk = {
      main = {
        type = "disk";
        device = "/dev/vda";
        content = {
          type = "gpt";
          partitions = {
            boot = {
              size = "1M";
              type = "EF02";
            };
            ESP = {
              size = "512M";
              type = "EF00";
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/boot";
                mountOptions = [ "umask=0077" ];
              };
            };
            root = {
              end = "-4G";
              content = {
                type = "filesystem";
                format = "ext4";
                mountpoint = "/";
                mountOptions = [ "noatime" ];
              };
            };
            swap = {
              size = "100%";
              content = {
                type = "swap";
                discardPolicy = "both";
                resumeDevice = false;
              };
            };
          };
        };
      };
      vdb = {
        type = "disk";
        device = "/dev/vdb";
        content = {
          type = "gpt";
          partitions = {
            root = {
              size = "100%";
              content = {
                type = "filesystem";
                format = "ext4";
                mountpoint = "/srv";
                mountOptions = [ "noatime" ];
              };
            };
          };
        };
      };
    };
  };
}
```
A simple configuration creating 3 partitions on vda, boot, root and swap, and 1 partition for data mounted to srv. I'll be serving longhorn from the /srv/longhorn directory and garage for s3 from the /srv/garage directory. 

Next would be the general system configuration. Since this is long and very custom, I'm just gonna link it here, and you can view this in [the repo](https://git.yadunut.dev/yadunut/nix/src/branch/main/systems/x86_64-linux/nut-gc2/default.nix).

Now that the configuration is done, I have secrets(such as my k3s token) that I want to transfer over to the new system. To cleanly do this, I can generate the host keys on this system, use that to rekey the secret, transfer over the host keys and then erase them from my system when complete. 

```sh
tmp_dir=$(mktemp -d)
mkdir -p "$tmp_dir/etc/ssh"
MACHINE_NAME="nut-gc2"

ssh-keygen -A -f "$tmp_dir"
for key in "$tmp_dir"/etc/ssh/ssh_host_*_key; do
    [ -f "$key" ] || continue
    echo "Updating comment on $key ..."
    ssh-keygen -q -c -C "root@$MACHINE_NAME" -f "$key"
done
echo "ssh key: $(cat $tmp_dir/etc/ssh/ssh_host_ed25519_key.pub)"
```

Now with the ssh key generated, we can copy that over into our secrets.nix, and use age to rekey the secret
```sh
agenix --rekey
```

With this rekeyed, we can get to infecting the system
```sh
nix run github:nix-community/nixos-anywhere -- --generate-hardware-config nixos-generate-config ./systems/x86_64-linux/nut-gc2/hardware-configuration.nix --flake ".#${MACHINE_NAME}" --extra-files "${tmp_dir}" --print-build-logs --build-on remote "root@${MACHINE_IP}"
```

Now with the server infected, you'll need to update your `~/.ssh/known_hosts` and you can then ssh into your machine