DiamantTh/installimage
1
0
Fork 0
Code Issues 16 Pull requests 2 Projects Releases Packages Wiki Activity Actions

Add support for installation with full disk encryption #21

Closed
overlordtm wants to merge 3 commits from luks into master
pull from: luks
merge into: DiamantTh:master
Conversation 15 Commits 3 Files changed 4 +91 -12
overlordtm commented 2019-10-15 21:14:12 +02:00 (Migrated from github.com)
Copy link

It is not yet finished, but opening PR if you have any feedback, because there is not much information in repository on "how to contribute" :)

PR contains changes needed to create server with encrypted root partition i.e. LVM on LUKS

You can specify lvm+luks value filesystem in PART

LUKS_PASSWORD test
FDE_SSH_UNLOCK 1 # set to 1 to enable remote unlock via SSH
PART /boot ext3 1G
PART lvm+luks   vg0   all

So far tested on Ubuntu 18.04 image on hCloud VM. If FDE_SSH_UNLOCK is set to 1, dropbear SSH server is started at port 54321 after boot and waits for connections. Password login is disabled, by default all SSH keys set for root user are also copied to dropbear.

It is not yet finished, but opening PR if you have any feedback, because there is not much information in repository on "how to contribute" :) PR contains changes needed to create server with encrypted root partition i.e. LVM on LUKS You can specify `lvm+luks` value filesystem in `PART` ``` LUKS_PASSWORD test FDE_SSH_UNLOCK 1 # set to 1 to enable remote unlock via SSH PART /boot ext3 1G PART lvm+luks vg0 all ``` So far tested on Ubuntu 18.04 image on hCloud VM. If FDE_SSH_UNLOCK is set to 1, dropbear SSH server is started at port 54321 after boot and waits for connections. Password login is disabled, by default all SSH keys set for root user are also copied to dropbear.
👍 6
ricariel commented 2019-10-18 10:48:43 +02:00 (Migrated from github.com)
Copy link

Thank you!

Thank you!
jamesponddotco commented 2019-11-25 03:59:07 +01:00 (Migrated from github.com)
Copy link

@overlordtm I was testing this on a dedicated server from Hetzner and it seems like it doesn't work on those — at least not when software RAID is used.

My autosetup file looks like this:

DRIVE1 /dev/nvme0n1
DRIVE2 /dev/nvme1n1
DRIVE3 /dev/nvme2n1
DRIVE4 /dev/nvme3n1

SWRAID 1
SWRAIDLEVEL 10

BOOTLOADER grub

HOSTNAME ubuntu1804

LUKS_PASSWORD test-test-test-test
FDE_SSH_UNLOCK 1

PART /boot ext4   1G
PART lvm+luks   vg0    all

LV vg0   logs   /var/logs    ext4         100G
LV vg0   loops  /dev/loops   ext4         1600G
LV vg0   home   /home        ext4         5G
LV vg0   tmp    /tmp         reiserfs     5G
LV vg0   root   /            ext4         all

IMAGE /root/.oldroot/nfs/install/../images/Ubuntu-1804-bionic-64-minimal.tar.gz

And this is how I ran your version of install image:

git clone --single-branch --branch luks https://github.com/overlordtm/installimage.git
alias installimage='/root/installimage/installimage'
installimage -c /root/autosetup -f yes -a

The install works until step 5/17 and then show this error message:

   5/17  :  Creating LVM volumes                            busy   Volume group "vg0" not found
  Cannot process volume group vg0
/root/installimage/functions.sh: line 2056: [: too many arguments
                                                           failed

When checking the debug.txt file, this seems to be the important part for the error:

[03:52:55] # Creating LVM volumes
[03:52:55] # Removing all Logical Volumes and Volume Groups
[03:52:55] # Removing all Physical Volumes
[03:52:55] # Creating PV /dev/md/1
[03:52:55] :   /dev/md/1: 6 bytes were erased at offset 0x00000000 (crypto_LUKS): 4c 55 4b 53 ba be
[03:52:55] :   /dev/md/1: 6 bytes were erased at offset 0x00004000 (crypto_LUKS): 53 4b 55 4c ba be
[03:52:55] # Creating encrypted PV /dev/md/1
[03:53:04] :   Name "md/1_crypt" invalid. It contains "/".
[03:53:04] :   Device /dev/mapper/md/1_crypt not found.
[03:53:04] # Creating VG vg0 with PV /dev/mapper/md/1_crypt
[03:53:05] :   Device /dev/mapper/md/1_crypt not found.
[03:53:05] # Creating LV vg0/logs (102400 MiB)
[03:53:05] :   Volume group "vg0" not found
[03:53:05] :   Cannot process volume group vg0
[03:53:05] => FAILED

I didn't look too much into it yet for lack of time, but I believe the problem happens on line 2000 of the functions.sh file, with the crypt_pv variable.

@overlordtm I was testing this on a dedicated server from Hetzner and it seems like it doesn't work on those — at least not when software RAID is used. My _autosetup_ file looks like this: ``` DRIVE1 /dev/nvme0n1 DRIVE2 /dev/nvme1n1 DRIVE3 /dev/nvme2n1 DRIVE4 /dev/nvme3n1 SWRAID 1 SWRAIDLEVEL 10 BOOTLOADER grub HOSTNAME ubuntu1804 LUKS_PASSWORD test-test-test-test FDE_SSH_UNLOCK 1 PART /boot ext4 1G PART lvm+luks vg0 all LV vg0 logs /var/logs ext4 100G LV vg0 loops /dev/loops ext4 1600G LV vg0 home /home ext4 5G LV vg0 tmp /tmp reiserfs 5G LV vg0 root / ext4 all IMAGE /root/.oldroot/nfs/install/../images/Ubuntu-1804-bionic-64-minimal.tar.gz ``` And this is how I ran your version of install image: ``` git clone --single-branch --branch luks https://github.com/overlordtm/installimage.git alias installimage='/root/installimage/installimage' installimage -c /root/autosetup -f yes -a ``` The install works until step 5/17 and then show this error message: ``` 5/17 : Creating LVM volumes busy Volume group "vg0" not found Cannot process volume group vg0 /root/installimage/functions.sh: line 2056: [: too many arguments failed ``` When checking the _debug.txt_ file, this seems to be the important part for the error: ``` [03:52:55] # Creating LVM volumes [03:52:55] # Removing all Logical Volumes and Volume Groups [03:52:55] # Removing all Physical Volumes [03:52:55] # Creating PV /dev/md/1 [03:52:55] : /dev/md/1: 6 bytes were erased at offset 0x00000000 (crypto_LUKS): 4c 55 4b 53 ba be [03:52:55] : /dev/md/1: 6 bytes were erased at offset 0x00004000 (crypto_LUKS): 53 4b 55 4c ba be [03:52:55] # Creating encrypted PV /dev/md/1 [03:53:04] : Name "md/1_crypt" invalid. It contains "/". [03:53:04] : Device /dev/mapper/md/1_crypt not found. [03:53:04] # Creating VG vg0 with PV /dev/mapper/md/1_crypt [03:53:05] : Device /dev/mapper/md/1_crypt not found. [03:53:05] # Creating LV vg0/logs (102400 MiB) [03:53:05] : Volume group "vg0" not found [03:53:05] : Cannot process volume group vg0 [03:53:05] => FAILED ``` I didn't look too much into it yet for lack of time, but I believe the problem happens on line 2000 of the _functions.sh_ file, with the _crypt_pv_ variable.
❤️ 1
overlordtm commented 2019-11-25 11:15:32 +01:00 (Migrated from github.com)
Copy link

Indeed it was never tested with root servers because I do not have a server to spare and I am too much of a cheapskate to spend setup free and monthly fee just to test it.

I think error is related to RAID, I was testing it with cloud instances (no raid there). If you still have server to install, I can take a look and try to push some fixes.

Probably the error originates from functions.sh:2000, where I try to strip /dev/ prefix to name crypt volume, but since underlying device is /dev/md/1 it ends up with invalid name md/1. There is anothe problematic substitution on line 2018

Indeed it was never tested with root servers because I do not have a server to spare and I am too much of a cheapskate to spend setup free and monthly fee just to test it. I think error is related to RAID, I was testing it with cloud instances (no raid there). If you still have server to install, I can take a look and try to push some fixes. Probably the error originates from functions.sh:2000, where I try to strip `/dev/` prefix to name crypt volume, but since underlying device is `/dev/md/1` it ends up with invalid name `md/1`. There is anothe problematic substitution on line 2018
overlordtm commented 2019-11-25 11:42:43 +01:00 (Migrated from github.com)
Copy link

@jamesponddotco I have created luks-root branch on my repo. The change is how I name LUKS devices, before I stripped /dev/ prefix and tried to name device with the remainder and _crypt suffix, so devices appeared something like /dev/mapper/sda1_crypt. Now I put crypt prefix in front and suffix is full device path with slashes relaced with underscores. So you should get something like /dev/mapper/crypt_md_0

Here is the commit
overlordtm/installimage@57c86b8cba

If you still have time to test, please report.

@jamesponddotco I have created luks-root branch on my repo. The change is how I name LUKS devices, before I stripped `/dev/` prefix and tried to name device with the remainder and `_crypt` suffix, so devices appeared something like `/dev/mapper/sda1_crypt`. Now I put `crypt` prefix in front and suffix is full device path with slashes relaced with underscores. So you should get something like `/dev/mapper/crypt_md_0` Here is the commit https://github.com/overlordtm/installimage/commit/57c86b8cbaa701d22cb4c42002acf22df936050d If you still have time to test, please report.
jamesponddotco commented 2019-11-25 14:33:03 +01:00 (Migrated from github.com)
Copy link

@overlordtm I ran a quick test and it seems to be working as expected 🎉

Might be worth adding a warning saying that ed25519 keys are not supported, though.

Nice work, man! Hope @asciiprod merge this soon :)

@overlordtm I ran a quick test and it seems to be working as expected 🎉 Might be worth adding a warning saying that _ed25519_ keys are not supported, though. Nice work, man! Hope @asciiprod merge this soon :)
jamesponddotco commented 2019-11-25 14:48:01 +01:00 (Migrated from github.com)
Copy link

@overlordtm One change I would make, mostly to keep everything easier, is to change authorized_keys and add this:

no-port-forwarding,no-agent-forwarding,no-X11-forwarding,command="/bin/cryptroot-unlock" KEY

This will show the cryptsetup password prompt directly, instead of the busybox prompt.

@overlordtm One change I would make, mostly to keep everything easier, is to change _authorized_keys_ and add this: ``` no-port-forwarding,no-agent-forwarding,no-X11-forwarding,command="/bin/cryptroot-unlock" KEY ``` This will show the cryptsetup password prompt directly, instead of the busybox prompt.
asciiprod commented 2019-11-25 15:24:52 +01:00 (Migrated from github.com)
Copy link

Thank you for your contribution. However, a feature should ideally be usable across all currently supported distros. Plus there are a number of edge cases (e.g. dropbear only works with keys, but not ed25519).

Also functions defined in functions.sh must either be working cross-distro or overloaded with distro-specific ones in their corresponding scripts. In the latter case the one in function.sh must throw an error (in case the distro specific script does not implement the function).

I should also mention, that we have (or have tried to) implemented this feature internally some time ago. The approach was a bit different in the details, but in general the same. However, we haven't merged it, for the reasons mentioned above.

Thank you for your contribution. However, a feature should ideally be usable across all currently supported distros. Plus there are a number of edge cases (e.g. dropbear only works with keys, but not ed25519). Also functions defined in functions.sh must either be working cross-distro or overloaded with distro-specific ones in their corresponding scripts. In the latter case the one in function.sh must throw an error (in case the distro specific script does not implement the function). I should also mention, that we have (or have tried to) implemented this feature internally some time ago. The approach was a bit different in the details, but in general the same. However, we haven't merged it, for the reasons mentioned above.
jamesponddotco commented 2019-11-25 15:30:00 +01:00 (Migrated from github.com)
Copy link

@asciiprod Would it be okay for me, as a customer, to keep a fork using the changes that @overlordtm made? We only run Ubuntu in our servers, so this should be okay — and I made a few modifications to his modifications anyway.

The LICENSE doesn't seem to mention a specific license, but it does say it can be used and modified freely — doesn't say anything about distributing it, though, and I would prefer to keep it in a public repository.

@asciiprod Would it be okay for me, as a customer, to keep a fork using the changes that @overlordtm made? We only run Ubuntu in our servers, so this should be okay — and I made a few modifications to his modifications anyway. The [LICENSE](https://github.com/hetzneronline/installimage/blob/master/LICENSE) doesn't seem to mention a specific license, but it does say it can be used and modified freely — doesn't say anything about distributing it, though, and I would prefer to keep it in a public repository.
asciiprod commented 2019-11-25 15:32:58 +01:00 (Migrated from github.com)
Copy link

Yes, you can do that. We should probably change that to something well-known license.

Yes, you can do that. We should probably change that to something well-known license.
👍 2
overlordtm commented 2019-11-25 15:52:59 +01:00 (Migrated from github.com)
Copy link

Hey @asciiprod, under official images you are thinking of Ubuntu, Debian, CentOS and OpenSUSE?

I submitted PR early, hoping someone else notices it and maybe contribute some ideas and feedback (like key type check ...), it is half finished fork :)

I had focus only on Ubuntu, as it is also our distribution of choice, but I believe I can make it work on all 4 distros mentioned before, if there is some interest on your side, otherwise we are perfectly happy with Ubuntu only. But would be nice to have FDE as out of box option at Hetzner.

BTW, I tested today changes on cloud instance (Ubuntu 18.04), still works :)

Hey @asciiprod, under official images you are thinking of Ubuntu, Debian, CentOS and OpenSUSE? I submitted PR early, hoping someone else notices it and maybe contribute some ideas and feedback (like key type check ...), it is half finished fork :) I had focus only on Ubuntu, as it is also our distribution of choice, but I believe I can make it work on all 4 distros mentioned before, if there is some interest on your side, otherwise we are perfectly happy with Ubuntu only. But would be nice to have FDE as out of box option at Hetzner. BTW, I tested today changes on cloud instance (Ubuntu 18.04), still works :)
asciiprod commented 2019-11-25 16:00:38 +01:00 (Migrated from github.com)
Copy link

I'll check if we can merge our generic LUKS code. You could then use a post-install script to install and configure dropbear

I'll check if we can merge our generic LUKS code. You could then use a post-install script to install and configure dropbear
❤️ 4
Phil-Friderici commented 2020-01-06 23:40:53 +01:00 (Migrated from github.com)
Copy link

@asciiprod Would it be possible for you to make your generic LUKS branch availalbe for the interessted public ? Beside getting feedback there is a chance to start working on a dropbear post-install :)

I've a use case and will test lvm+luks+dropbear in combination with Debian 10.x.

@asciiprod Would it be possible for you to make your generic LUKS branch availalbe for the interessted public ? Beside getting feedback there is a chance to start working on a dropbear post-install :) I've a use case and will test lvm+luks+dropbear in combination with Debian 10.x.
asciiprod commented 2020-09-08 15:22:35 +02:00 (Migrated from github.com)
Copy link

If have just merged our encryption support. No batteries included (initramfs-dropbear or clevis/tang), but these are easily added via post-install script.

If have just merged our encryption support. No batteries included (initramfs-dropbear or clevis/tang), but these are easily added via post-install script.
gnat commented 2021-02-22 10:38:51 +01:00 (Migrated from github.com)
Copy link

The official way needs to have a sample post-install script included in the ./post-install directory and how to execute it @asciiprod because currently this is far too obscure compared to @overlordtm solution. While we all appreciate your efforts, zero documentation and zero examples for a script so specific to Hetzner is not acceptable.

The official way needs to have a sample post-install script included in the `./post-install` directory and how to execute it @asciiprod because currently this is far too obscure compared to @overlordtm solution. While we all appreciate your efforts, zero documentation and zero examples for a script so specific to Hetzner is not acceptable.
gnat commented 2021-02-22 10:49:45 +01:00 (Migrated from github.com)
Copy link

There needs to be a more polished way to do this in 2021. GDPR and new data protection laws put a lot of pressure on us to encrypt data at rest. @asciiprod Encryption at rest is the norm at other providers such as AWS and Google Cloud.

There needs to be a more polished way to do this in 2021. GDPR and new data protection laws put a lot of pressure on us to encrypt data at rest. @asciiprod Encryption at rest is the norm at other providers such as AWS and Google Cloud.

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

   
duplicate

   
enhancement

   
help wanted

   
invalid

   
not a bug

   
question

   
wontfix
No labels
bug
duplicate
enhancement
help wanted
invalid
not a bug
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
DiamantTh/installimage!21
No description provided.