11.25.14
Posted in Technology at 08:06:24 by Jeff Hardy
Projects evolve, code boundaries change. Git commit history shall be preserved.
Split Repo
Split files out into a new repo preserving all history. Existing repository: perl. File to preserve: psntrophy.pl.
First, clone the existing repository to a new location, as it will be carved down to just the file(s) we need, thereby becoming the new desired repository:
cd tmp
git clone ~/workspace/perl
cd perl
git remote remove origin
Remove everything but the matching file by use of filter-branch. There are other approaches, particularly for subdirectories, but I found this to work best (all one line):
git filter-branch -f --prune-empty
--index-filter 'git ls-tree -z --name-only --full-tree HEAD |
grep -zv "^psntrophy.pl$" |
xargs -0 git rm --cached -r -f --ignore-unmatch' -- --all
Rebase to eliminate commits related to renames that are not related to this file, and only remain because git ls-tree did not find these non-existent files for our filter:
git rebase --onto bf1fbad2b6f766b80c2a07edacd7084c6d3fd252 e490b6cb41ca541b7e468148a62f63435366836f
git rebase --onto 31fe475fe0c7314a2f307fa1c201cd2f9eca6992 bf1fbad2b6f766b80c2a07edacd7084c6d3fd252
Garbage collection on 694 objects at 4MB:
git gc
Counting objects: 694, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (674/674), done.
Writing objects: 100% (694/694), done.
Total 694 (delta 253), reused 0 (delta 0)
du -csh .git/objects/
592K .git/objects/
592K total
Merge Repos
Merge several repositories, each into subdirectories of a new repository, preserving all history. New repository: unix. Project to merge: ccli.
First, clone the project we wish to merge in so as to preserve the original in case something goes awry:
cd tmp
git clone ~/workspace/ccli
cd ccli
git remote remove origin
cd ..
Now back at top-level tmp, create a new target repository and start with a an initial dummy commit
git init unix
cd unix
echo "initial" > deleteme.txt
git add deleteme.txt
git commit -m "Initial commit."
git rm ./deleteme.txt
git commit -m "Initial delete."
Add a remote to the ccli repo:
git remote add ccli ../ccli
git fetch ccli
git merge ccli/master
Move files and commit:
mkdir ccli
for i in $(ls -1 | grep -v ccli); do git mv $i ccli; done
git mv .ccli_env_example ccli
git commit -m "Move ccli project into ccli subdir"
git remote remove ccli
Repeat for each project to merge in.
NOTE: Alternative is to fetch each project into separate branches, merge the branches. Cleaner, but end result same.
NOTE: Requires –follow flag to track full file history in git log, as this is a move like any other. There may be a history rewrite alternative.
NOTE: Approaches with submodules keep subdir distinct so future changes from upstream can be brought in. This is not what we want, as we wish to glue things together permanently.
Upload
Should we desire to share our work by pushing it to a new remote repository, such as GitHub:
git remote add origin https://github.com/fritzhardy/psnextract.git
git push -u origin master
Resources
Permalink
11.11.14
Posted in Technology at 07:15:12 by Jeff Hardy
Having worked with VirtualBox for some time, I recently went through a physical-to-virtual migration of my bare-metal Fedora install (in parallel to its hardware migration). The process for a P2V migration is quite straightforward.
Start with a dd of the source drive or drives (entire drive, not partitions):
dd if=/dev/sda of=/storage/fritzdesk_sda
The image must be converted to VDI format before it can be used with a VirtualBox virtual machine, and the VBoxManage command does the trick:
VBoxManage convertdd fritzdesk_sda fritzvbox-disk1.vdi --format VDI
Converting from raw image file="fritzdesk_sda" to file="fritzvbox-disk1.vdi"...
Creating dynamic image with size 40037760000 bytes (38183MB)...
If this image will be associated with a new VM, it seems best to create a bare-bones VM with no disk first. Doing so will create the underlying folder structure within which the image can be copied, and then incorporated into the VM. Otherwise, VirtualBox will be perfectly happy to use the image wherever it lies on the filesystem, a recipe for losing track of things.
The VBoxManage command is quite powerful, and capable of a number of image operations. Another common operation, as we move data around within the VM and storage needs change, is disk resize:
VBoxManage modifyhd fritzvbox-disk1.vdi --resize 50000
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
As anywhere, image operations outside the VM will need to be accompanied by disk operations within: pvresize, lvresize, resize2fs, etc.
As for this particular P2V operation, there were but few changes needed for successful boot, namely the removal of VirtualBox host packages such as kmod-VirtualBox (it was once a host itself), and adjusting some network settings. Beyond that, storage re-arrangement akin to operations done in parallel hardware migration, here with the added ease of being a virtual machine.
Permalink
08.17.14
Posted in Technology at 15:18:38 by Jeff Hardy
Encrypted swap is at least as important as encrypted data. This is because the contents of nearly anything in memory can be swapped to disk, at any time, and for any length of time. That includes passwords and like, en claire. Encrypting the system swap volume cannot be overlooked.
During our migration process we have cloned drives in part one, switched to an encrypted root in part two, and moved partitions around to make room for home data encryption in part three. Now we will encrypt the swap partition.
Goals
We are free of many of the concerns with which we dealt when dealing with the encrypted root in part two, but we have two key goals:
- Support for hibernate
- Avoid multiple passphrase entry at boot without compromising security
The first is a property unique to swap, that being the place were the contents of RAM are copied when a machine is put into hibernation. Our encryption setup should support that. The second is an artifact of our disk layout, this being a second encrypted volume to be encountered during boot. There are more than a few ways to meet both of these goals, and some shortcuts that can be taken should one or the other not be desired.
Suspend vs Hibernate
Imprecision characterizes the names and labels for “hardware sleep” as it does many areas of technology, even after the ideas and processes have been codified. It is worth stating a couple of definitions:
- Suspend: Processes paused, most parts of the computer turned off except for RAM in order to save and resume machine state (aka “Sleep” or “Standby”)
- Hibernate: Processes paused, memory flushed to swap area of the disk before complete shutdown in order to save and resume machine state from full poweroff/poweron (aka “Suspend-to-disk”)
We pursue the latter, as that is all that is relevant for a swap change.
Possibilities
There are a number approaches for volume encryption, most of which we did not explore in parts two and three. All boot-time decryption is handled by systemd-cryptsetup-generator, which understands /etc/crypttab entries as well as the rd.luks kernel command-line arguments we used previously.
- encrypted physical volume (not available with our chosen volume layout)
- crypttab swap entry with random key
- crypttab volume entry assuming passphrase
- crypttab volume entry specifying key stored on some device
- kernel rd.luks argument assuming passphrase entry
- kernel rd.luks argument specifying key stored on some device
Option #1 is not available without redoing this particular storage layout from scratch, but would offer the advantage that all logical volumes atop a crypto-pv are implicitly encrypted: root, swap, whatever. This route was not chosen simply because it did not seem to align with install-time Fedora defaults.
Options #2, #3, #4 make use of /etc/crypttab, something we did not explore when setting up encrypted root and home, and entries here are generally set up by systemd later in the boot process after initrd. In the entries below, the first is the simplest means of encrypting swap, and a facility unique to crypttab: recreating it with random key at each boot, leaving a plain-crypt (not LUKS) device mapper entry named crypt-swap. The latter two point to a LUKS volume that would need to be cryptsetup manually ahead of time, with masochistically named device-mapper entries in the form luks-UUID:
crypt-swap /dev/mapper/Volume00-swap /dev/urandom swap
luks-XXX-YYY-ZZZ-AAA-BBB-CCC /dev/mapper/Volume00-swap
luks-XXX-YYY-ZZZ-AAA-BBB-CCC /dev/mapper/Volume00-swap /etc/luks/swap.key
Out of this bunch, the first does not support hibernate by nature of its use of random key. But #2 and #3 are viable, with concerns about key storage for the latter assuaged by its location within the encrypted root.
Options #5 and #6 are the way we handled root in part two, for decryption by the initial ram disk. This would add another rd.luks.uuid kernel argument in grub.conf pointing to the device in question:
rd.luks.uuid=XXX-YYY-ZZZ-AAA-BBB-CCC
rd.luks.uuid=XXX-YYY-ZZZ-AAA-BBB-CCC rd.luks.key=/path/to/swap.key
Out of these two, the second seems to succumb to problems with systemd within the initial ramdisk being able to make use of the argument, exacerbated by myriad documentation conflicts between the use of this option beneath the init/upstart and systemd regimes. This post contains a good explanation and possible workaround, but even should it work, there is the problem of where to store a key, since the encrypted root would not yet be available. That leaves #5 viable.
With some of our options relying on passphrase entry, entering multiple phrases one after another could be a problem. It turns out that this is not a concern as long as we adhere to one condition: use of the same passphrase. Though it is not documented well (or at all), passphrases and keys are cached during systemd cryptsetup (by plymouth) for attempt against every encrypted volume. With the same one in use on both, the end result is one prompt at boot time.
Three options meet our two goals: #2, #3, #5. All work, but we select #5. Because we already have the initial ramdisk decrypting root by passphrase, it makes some bit of sense to do the same for swap at the same stage. Also, this approach means there are no concerns about key storage. And lastly, specifying UUIDs on the kernel command line for luks means that it could still work in the event a default ramdisk is used.
Setup Encrypted Swap
Our disk layout is as follows:
/dev/sda (150GB)
sda1: ext3 /boot (300MB)
sda2: lvm Volume00 (150GB)
Volume00-swap: swap (2GB)
Volume00-root: LUKS (127GB)
luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 ext3 (127GB)
unallocated: (19GB)
We will simply re-use the existing swap logical volume. Though encrypting an existing volume destroys data, that of the swap partition is normally of no lasting consequence.
First we need to disable swap:
swapoff -a
Write random data to the volume for best results:
dd if=/dev/urandom of=/dev/mapper/Volume00-swap
dd: writing to ‘/dev/mapper/Volume00-swap’: No space left on device
4194305+0 records in
4194304+0 records out
2147483648 bytes (2.1 GB) copied, 567.601 s, 3.8 MB/s
Encrypt the volume:
cryptsetup --verbose --verify-passphrase luksFormat /dev/mapper/Volume00-swap
WARNING!
========
This will overwrite data on /dev/mapper/Volume00-swap irrevocably.
Are you sure? (Type uppercase yes): YES
Enter passphrase:
Verify passphrase:
Command successful.
Grab the volume UUID:
cryptsetup luksDump /dev/mapper/Volume00-swap | grep UUID
UUID: 8f747652-3e73-4297-9721-9e8cdd8d89b1
Masochistically open the volume at name reflective of the UUID:
cryptsetup luksOpen /dev/mapper/Volume00-swap luks-8f747652-3e73-4297-9721-9e8cdd8d89b1
Enter passphrase for /dev/mapper/Volume00-swap:
Make swap:
mkswap /dev/mapper/luks-8f747652-3e73-4297-9721-9e8cdd8d89b1
Setting up swapspace version 1, size = 2095100 KiB
no label, UUID=35d0810f-accb-4fc8-ad9c-3c88a28429ad
Edit /etc/fstab to reflect the new swap location:
/dev/mapper/luks-8f747652-3e73-4297-9721-9e8cdd8d89b1 none swap defaults 0 0
Turn on swap to ensure all is well:
swapon -a
swapon -s
Filename Type Size Used Priority
/dev/dm-2 partition 2095100 0 -1
With working swap, we need to ensure it comes up on boot.
Edit /etc/default/grub to add this new rd.luks.uuid and make config:
GRUB_CMDLINE_LINUX="quiet rd.luks.uuid=cc7e78ee-87a9-4ad0-9c82-31de01389b34 rd.luks.uuid=luks-8f747652-3e73-4297-9721-9e8cdd8d89b1"
grub2-mkconfig -o /boot/grub2/grub.cfg
Create new ramdisk so it can find the new volume:
cd /boot
cp -a initramfs-3.12.8-300.fc20.i686.img initramfs-3.12.8-300.fc20.i686.img.20140817
dracut -f /boot/initramfs-3.12.8-300.fc20.i686.img 3.12.8-300.fc20.i686
Boot with New Swap
At this point, we can boot with the new crypto swap in place, and our system disk is as follows:
/dev/sda (150GB)
sda1: ext3 /boot (300MB)
sda2: lvm Volume00 (150GB)
Volume00-swap: swap (2GB)
luks-8f747652-3e73-4297-9721-9e8cdd8d89b1 swap (2GB)
Volume00-root: LUKS (127GB)
luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 ext3 (127GB)
unallocated: (19GB)
Now everything is truly encrypted.
Permalink
07.10.14
Posted in Technology at 22:55:42 by Jeff Hardy
GParted is among the most well-known and respected open-source partition editors. From creation to deletion, resizing, copying, and moving, it is a tool built to manage the full gamut of disk partitionining operations.
In part one of our migration process, we cloned the installation to a new drive in new hardware. In part two, we moved to a LUKS-encrypted root. Now we have a few adjustments we wish to make to our partitioning scheme, regaining space so that we may move the rest of the system beneath the umbrella of encryption.
The Plan
We have the following arrangement on the system drive:
/dev/sda (150GB)
sda1: ext3 /boot (300MB)
sda2: ext3 oldroot (38GB)
sda3: lvm Volume00 (112GB)
Volume00-home: ext3 /home (70GB)
Volume00-swap: swap (2GB)
Volume00-root: LUKS (40GB)
luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34: / ext3 (40GB)
Despite our efforts thus far, the issue remains that our home volume is not yet protected by encryption. We can tackle this two ways, either by building a new home crypto-volume and copying data as we did for root, or by simply reclaiming space enough to abandon a separate home volume and copy that data beneath the already encrypted root. I elect for the latter, mainly because the use of a single crypto-volume avoids the potential for multiple pass-phrase entry at boot, and also because there is no compelling reason for a separate home on this particular system.
In order to gain the space necessary, we will remove the old unused root, add that space for use by LVM, grow our new crypto-root, and then copy the data from home. This is no big deal, and the partition is easily deleted with fdisk or gparted. But there is one issue. Because this is not the last partition on the disk, resizing for our needs is not as simple as an fdisk delete and recreate, moving the end boundary to the end of the drive. Now we need to move the start of the partition. Enter GParted.
GParted
While GParted can be run at any time, some functionality for a given storage device may not be available, nor entirely safe, if it is in-use (mounted) by the operating system. For this reason, we elect to boot the system with the GParted Live CD.
NOTE: Be aware that there is a perfectly good alternative to partition resizing of any kind. We could instead elect to create an LVM partition using the unallocated space and do standard LVM operations to incorporate it into our root logical volume (pvcreate, vgextend, lvextend). But we continue with full confidence in GParted, the main benefit being mostly cosmetic in the end: a tidy partition table.
Booted into the live environment, we start GParted. After deleting the old root partition at /dev/sda2, we are left with a good illustration of the situation at hand. We have a region of unallocated space before the LVM partition:
We need to resize/move the sda3 partition, which contains our priceless crypto-LVM/root within it:
The resize dialog gets right to the point. Adjusting the amount of space before and after the partition moves its start and end positions on disk:
Drag the start boundary all the way to the left (or enter 0) to effect a move/resize, absorbing every bit of the unallocated space before the partition:
Operation pending. We can still back out without affecting anything. Click apply, and be prepared for a lengthy wait.
When all is said and done, we have moved and resized our LVM partition to utilize the full disk:
Note that used space has not changed. We address that next.
Resize LVM, LUKS, Ext3
The work done so far has resized the outermost container, the partition. Like Russian nesting dolls, we need to grow each in turn before we can address the filesystem. A testament to the tools, we can actually do all of the following booted into the live system.
First LVM:
pvs
PV VG Fmt Attr PSize PFree
/dev/sda3 Volume00 lvm2 a-- 148.75g 36.98g
vgs
VG #PV #LV #SN Attr VSize VFree
Volume00 1 3 0 wz--n- 148.75g 36.98g
lvs
LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert
home Volume00 -wn-ao---- 69.00g
root Volume00 -wi-ao---- 40.76g
swap Volume00 -wi-ao---- 2.00g
lvresize Volume00/root -l+100%FREE
Extending logical volume root to 77.75 GiB
Logical volume root successfully resized
lvs
LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert
home Volume00 -wn-ao---- 69.00g
root Volume00 -wi-ao---- 77.75g
swap Volume00 -wi-ao---- 2.00g
NOTE: I would normally expect to first do a pvresize to grow the PV, but the PV already knew about the extra space, likely a gift from GParted.
Then LUKS:
cryptsetup resize luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34
echo $?
0
Finally the filesystem:
df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 40G 29G 9.2G 76% /
resize2fs /dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34
resize2fs 1.42.8 (20-Jun-2013)
Filesystem at /dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 is mounted on /; on-line resizing required
old_desc_blocks = 3, new_desc_blocks = 5
The filesystem on /dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 is now 20380160 blocks long.
df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 77G 29G 45G 40% /
We are left with a nice large root to accommodate our /home data.
Home Copy
With 45GB free in our root, we should have plenty of space after cleaning out some garbage in home:
df -h /home
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/Volume00-home 64G 5.0G 56G 9% /home
A good old rsync will suffice to copy that whopping 5GB of data from the existing home volume to the home within the crypto-root. First things first, bug out to runlevel 3 (or single/rescue mode for overkill) to ensure home is not in use so we can mount it somewhere else:
init 3
umount /home
mkdir /oldhome
mount /dev/mapper/Volume00-home /oldhome
Finally rsync the data:
rsync -a /oldhome/ /home
When done, remember to pound out the /home mount from /etc/fstab lest we accidentally mount our old unencrypted home volume on next boot:
#/dev/Volume00/home /home ext3 defaults 1 2
A reboot confirms that everything is working as it should.
LVM Cleanup
Almost half the disk is tied up in our abandoned home. Once happy with the new arrangement, we can easily reclaim that space starting at the LVM layer, and then working our way up. This is just a second run at container resizing as above, LVM, LUKS, Ext3, all live.
lvs
LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert
home Volume00 -wn-a----- 69.00g
root Volume00 -wi-ao---- 77.75g
swap Volume00 -wi-ao---- 2.00g
lvremove Volume00/home
Do you really want to remove active logical volume home? [y/n]: y
Logical volume "home" successfully removed
vgs
VG #PV #LV #SN Attr VSize VFree
Volume00 1 2 0 wz--n- 148.75g 69.00g
lvextend -L+50G Volume00/root
Extending logical volume root to 127.75 GiB
Logical volume root successfully resized
cryptsetup resize luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34
echo $?
0
resize2fs /dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34
resize2fs 1.42.8 (20-Jun-2013)
Filesystem at /dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 is mounted on /; on-line resizing required
old_desc_blocks = 5, new_desc_blocks = 8
The filesystem on /dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 is now 33487360 blocks long.
df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 126G 36G 85G 30% /
NOTE: With ample disk available, we left some space unallocated in the VG. This is just a hedge for flexibility. If we need more space in the future, we can always add it live just as we did here.
Partition Re-Numbering
Our partition table consists of the following, as revealed by fdisk:
Device Boot Start End Blocks Id System
/dev/sda1 * 63 619919 309928+ 83 Linux
/dev/sda3 620544 312580095 155979776 8e Linux LVM
While it is no problem whatsoever to be “missing” a partition in sequence, we continue the intrepid pursuit of the trivial and wish to renumber sda3 to sda2. The easiest way to rectify this is to simply delete and re-create the partition using fdisk. Since it is the last partition, fdisk will normally default to the same start and end sectors when creating the new one (something to double-check). While this can be done live followed by an immediate reboot, it is safest to do this from rescue mode with the disk not in use.
fdisk /dev/sda
ddelete, partition number 3
new, primary, partition number 2, first sector default, last sector default
type, partition number 2, 8e
write
Reboot. Just a reminder, this is completely unnecessary. 
Everything Encrypted
We leave with this on the system disk:
/dev/sda (150GB)
sda1: ext3 /boot (300MB)
sda2: lvm Volume00 (150GB)
Volume00-swap: swap (2GB)
Volume00-root: LUKS (127GB)
luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 ext3 (127GB)
unallocated: (19GB)
A tidy partition and volume layout with nearly everything encrypted. In part four, we will contend with encrypting swap and explore encryption possibilities more fully.
Permalink
06.21.14
Posted in Technology at 23:23:55 by Jeff Hardy
Dracut is an initramfs infrastructure, as well as the name of tool needed to create initramfs images. It is also the name of the prompt/environment made available at boot time by choice or by error, allowing for troubleshooting of boot-time problems.
Building Initramfs Images
It always pays to make backups:
cd /boot
cp -a initramfs-3.12.8-300.fc20.i686.img initramfs-3.12.8-300.fc20.i686.img.backup
The dracut tool is used to build initramfs images:
dracut -f /boot/initramfs-3.12.8-300.fc20.i686.img 3.12.8-300.fc20.i686
Images are best built from within the intended root of the Linux system, and the tool can be run on a live system with no issues. If the system is in a state where this is unattainable, it is possible to build a clean environment by assembling the necessary mounts in rescue mode. Do not select “find drives automatically,” we will do so manually:
mkdir /mnt/sysimage
vgchange -ay
mount /dev/mapper/Volume00-root
mount /dev/sda1 /mnt/sysimage/boot
mount --bind /proc /mnt/sysimage/proc
mount --bind /dev /mnt/sysimage/dev
mount --bind /sys /mnt/sysimage/sys
chroot /mnt/sysimage
NOTE: Devices holding boot and root will vary from system to system.
Now run dracut.
The Dracut Shell
If the boot process fails due to an issue with the initial ram disk, we will likely be dropped to the dracut shell (this may require adding rd.shell as a kernel argument).
For example, failure due to a missing volume:
Warning: Could not boot.
Warning: /dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 does not exist
The beauty of the dracut prompt is that we have nearly the full flexibility of Linux at our disposal, with whatever tools exist in the initramfs image.
This particular error is something I have commonly experienced when switching root, and is probably due to the systemd setup within the initramfs specifically looking for a device that did not exist when we created it (for instance, if we ran dracut while not within the final resting place of root).
We can see what is actually contained within the initrams image:
lsinitrd /boot/initramfs-3.12.8-300.fc20.i686.img
If a volume cannot be found, it is likely there is no entry for it in etc/systemd/system/initrd.target.wants. We can attempt to urge things along manually:
lvm vgchange -ay
cryptsetup luksOpen /dev/mapper/Volume00-root luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34
Enter passphrase for /dev/mapper/Volume00-root:
exit
If this now boots the system, the path forward is to re-generate the initramfs image correctly, which usually entails doing so from within the appropriate root (as above).
Finally, here is a tidbit showing what kinds of things can be done for gathering intelligence for later followup:
debug=$(lvm pvdisplay; lvm vgdisplay; lvm lvdisplay; blkid; dmesg)
lvm vgchange -a y
cryptsetup luksOpen /dev/mapper/Volume00-root luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34
Enter passphrase for /dev/mapper/Volume00-root:
mount /dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 /mnt
echo "$debug" > /mnt/dracut.debug
NOTE: All this does is set an environment variable “debug” to the output of a bunch of commands, then activate storage somewhere to store that output, in this case showing how it is even possible to use a crypto-volume. Perhaps it is USB or simple volume instead.
Resources
- http://wiki.centos.org/TipsAndTricks/CreateNewInitrd
- http://lists.centos.org/pipermail/centos/2013-March/133604.html
Permalink
Posted in Technology at 23:22:23 by Jeff Hardy
GNU GRUB is the bootloader of choice for most Linux distributions, offering massive flexibility to boot a multitude of operating systems. With GRUB version 2, configuration remains straightforward, but can be largely auto-generated with modern tools. It also retains the full flexibility to modify the boot configuration at run-time.
Configuration
It always pays to make backups:
cd /boot/grub2
cp -a grub.cfg grub.cfg.backup
In order to generate grub2 configuration:
grub2-mkconfig -o /boot/grub2/grub.cfg
The tool will probe the system looking for partitions containing vmlinuz and initramfs images, and build configuration appropriately. Administrator tunables that affect the process are stored in /etc/default/grub. Example config:
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
# GRUB_TERMINAL="serial console"
# GRUB_SERIAL_COMMAND="serial --unit=0 --speed=9600"
GRUB_CMDLINE_LINUX="quiet rhgb"
GRUB_DISABLE_RECOVERY="true"
Configuration is best built from within the intended root of the Linux system, and the tool can be run on a live system with no issues. If the system is in a state where this is unattainable, it is possible to build a clean environment by assembling the necessary mounts in rescue mode. Do not select “find drives automatically,” we will do so manually:
mkdir /mnt/sysimage
vgchange -ay
mount /dev/mapper/Volume00-root
mount /dev/sda1 /mnt/sysimage/boot
mount --bind /proc /mnt/sysimage/proc
mount --bind /dev /mnt/sysimage/dev
mount --bind /sys /mnt/sysimage/sys
chroot /mnt/sysimage
NOTE: Devices holding boot and root will vary from system to system.
Now run grub2-mkconfig.
Run-Time Changes
Hitting “e” at the grub menu allows one to peruse and change configuration. This is probably most commonly used to alter kernel command-line arguments, such as adding “single” to boot to single-mode, but any aspect of the configuration can be changed before boot.
For example, we may wish to attempt to boot an alternate root volume. While in this mode, we need simply find the kernel we wish to boot and alter its command-line arguments, one being the selection of the intended root:
linux /vmlinuz-3.12.8-300.fc20.i686 ro root=/dev/sda2
NOTE: Add/remove other arguments such as quiet, rhgb, or rd.* as necessary.
We may also have a need to load an alternate initramfs image:
initrd /initramfs-3.12.8-300.fc20.i686.img.backup
When satisfied with the changes, hit F10 to boot.
Grub Prompt
It is actually entirely possible to attempt boot with no grub configuration at all, whether dropping voluntarily or involuntarily to the grub prompt. Given all the same assumptions:
grub> ls
(hd0) (hd0,1) (hd0,2) (hd0,3)
grub> set root=(hd0,1)
grub> linux /vmlinuz-3.12.8-300.fc20.i686 ro root=/dev/sda2
grub> initrd /initramfs-3.12.8-300.fc20.i686.img.20140608
grub> boot
NOTE: The partition nomenclature is different in grub, but we are specifying the partition containing vmlinuz and initrd images (eventually known as sda1 and mounted at /boot by linux). Also note that tab-completion thankfully works.
Resources
- http://tuxers.com/main/instigating-a-manual-boot-from-the-grub-prompt
Permalink
Posted in Technology at 23:21:07 by Jeff Hardy
Full-disk encryption is essentially a requirement for a mobile OS. While there are a few choices for full block device encryption on Linux (such as Truecrypt), at this time the clear choice is Linux Unified Key Setup. LUKS integrates with dm-crypt as the encryption backend, producing a device mapper entry for the decrypted block device that can be used like any other.
In part one of our Fedora migration odyssey, we cloned the installation to a laptop. Now in part two, we will shift things around to move to a LUKS-encrypted root. What follows assumes that we are using the new laptop for all operations, and that our cloned system drive is the only one in place (sda). While everything can be done in rescue mode, some operations require it and will be noted.
The Plan
We have the following partition and volume layout:
/dev/sda (150GB)
sda1: ext3 /boot (300MB)
sda2: ext3 / (38GB)
sda3: lvm Volume00 (112GB)
Volume00-home: ext3 /home (70GB)
Volume00-swap: swap (2GB)
unallocated: (40GB)
The official documentation indicates that there is no way to encrypt-in-place and thereby preserve data (although tools such as luksipc suggest otherwise). Regardless, we will use that unallocated space, perfectly sized, to create a new root filesystem on top of a crypto-volume using LVM/LUKS, and then copy data. A move provides a secondary benefit here: it will fold root back into LVM, its excursion to a bare partition being only ever a mere workaround.
Last but not least, a copy leaves our existing root undisturbed as a fallback in the event of problems.
Create a New Crypto Volume
First we create an LVM volume using all of that free space:
vgs
VG #PV #LV #SN Attr VSize VFree
Volume00 1 2 0 wz--n- 111.76g 40.76g
lvcreate -n root -l+100%FREE Volume00
vgs
VG #PV #LV #SN Attr VSize VFree
Volume00 1 3 0 wz--n- 111.76g 0
lvs
LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert
home Volume00 -wn-ao---- 69.00g
root Volume00 -wi-ao---- 40.76g
swap Volume00 -wi-ao---- 2.00
Rather than put a filesystem directly on the new volume, we will set up for encryption. For the best encryption results, it is recommended to write random data to the device:
dd if=/dev/urandom of=/dev/mapper/Volume00-root
dd: writing to ‘/dev/mapper/Volume00-root’: No space left on device
85483521+0 records in
85483520+0 records out
43767562240 bytes (44 GB) copied, 11330.7 s, 3.9 MB/s
NOTE: This will take some time.
Encrypt the volume:
cryptsetup --verbose --verify-passphrase luksFormat /dev/mapper/Volume00-root
WARNING!
========
This will overwrite data on /dev/mapper/Volume00-root irrevocably.
Are you sure? (Type uppercase yes): YES
Enter passphrase:
Verify passphrase:
Command successful.
NOTE: Remember that passphrase. If lost, data will be irrecoverable.
Review the encryption summary for the device, noting the UUID:
cryptsetup luksDump /dev/mapper/Volume00-root
...
cryptsetup luksDump /dev/mapper/Volume00-root | grep UUID
UUID: cc7e78ee-87a9-4ad0-9c82-31de01389b34
Open the crypto-volume. The last argument specifies the device-mapper name by which this volume will be opened:
cryptsetup luksOpen /dev/mapper/Volume00-root luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34
NOTE: Naming by UUID is how Fedora sets these volumes up when creating them at install time, practice we will adhere to later when setting up the new root. For consistency’s sake, we do so now, but we could just as easily call it “luks-root” or some such for convenience at this stage.
Create a filesystem:
mkfs -t ext3 /dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34
Copy Data
At this point, we should boot to rescue mode if not already. Because we are copying the existing root volume, we cannot afford to have it mounted at / where it could change. A read-only remount could suffice, but is likely to cause issues on a live system. Additionally, though we tell rsync not to cross filesystem boundaries anyway (-x), it is best reduce to zero the risk of copying special fileystems or other partitions that may be mounted beneath root.
We mount the source and destination, and rsync the contents:
mkdir /oldroot
mount /dev/sda2 /oldroot
mkdir /newroot
vgchange -ay
cryptsetup luksOpen /dev/mapper/Volume00-root luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34
mount /dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 /newroot
rsync -avx /oldroot/ /newroot
Requirements for Switching to a New Root
There are three elements necessary for successful switch to a new root, be it partition, lvm, crypto-volume, floppy, or who knows what else. In the order they are encountered in the boot process:
-
GRUB: The bootloader needs to be configured properly to load the appropriate kernel executable with necessary arguments indicating where to find the new root.
-
Initial RAM Disk: The initramfs image (or older initrd) needs to be built with enough utility so that the kernel can make use of it to find the real root filesystem. It may or may not already be capable, depending on suitable kernel arguments specified in grub.
-
fstab: Configuration in /etc/fstab must be set to mount the new root at / with suitable mount options.
It is certainly possible to attempt to fight through each stage of the boot process, making run-time manual changes to grub and the dracut environments in order to find one’s way into the new root, confirming it works before making configuration changes. This entails having a decent amount of intelligence about how various rd.* kernel arguments will relate to one’s system. This is definitely a worthy endeavor and is certainly the approach to take when troubleshooting. But in the end, we will proceed directly to making the system changes for the new root, first making backups of the involved files, thereby allowing us to use the original root as a fallback without too much trouble.
Configure the System for a New Root
The modern tools necessary to build the bootloader config (grub2-mkconfig) and initramfs image (dracut), do a wonderful job of auto-configuring themselves. This is both a blessing and a curse, as we need to carefully establish an environment from which they can reliably draw their bits. This means running them from within the new root for best results, necessitating a boot to rescue mode.
Boot to rescue mode and make sure not to select “find drives automatically.” Instead, we will assemble what we need manually. The following is the surest path I have found to setup an environment for building clean grub configuration and initramfs image:
mkdir /mnt/sysimage
vgchange -ay
cryptsetup luksOpen /dev/mapper/Volume00-root luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34
Enter passphrase for /dev/mapper/Volume00-root:
mount /dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 /mnt/sysimage
mount /dev/sda1 /mnt/sysimage/boot
mount --bind /proc /mnt/sysimage/proc
mount --bind /dev /mnt/sysimage/dev
mount --bind /sys /mnt/sysimage/sys
chroot /mnt/sysimage
NOTE: If this was not a crypto-volume, the procedure still stands, mounting the LVM volume directly.
We have assembled the system from new root up, and are now prepared to re-configure and rebuild.
Make backups of the initramfs image and grub configuration:
cd /boot
cp -a initramfs-3.12.8-300.fc20.i686.img initramfs-3.12.8-300.fc20.i686.img.20140608
cd /boot/grub2
cp -a grub.cfg grub.cfg.20140608
NOTE: Grub configuration problems are relatively easy to work around at boot time, especially with a backup of the initramfs. Initramfs-specific problems less so. The initramfs may also be difficult to restore to original state depending on the nature of the problem. In other words, make sure to back up the initramfs.
Build the new initramfs image:
dracut -f /boot/initramfs-3.12.8-300.fc20.i686.img 3.12.8-300.fc20.i686
Alter grub configuration at /etc/default/grub, adding an argument informing our kernel that it should open our new crypto-volume:
GRUB_CMDLINE_LINUX="quiet rhgb rd.luks.uuid=cc7e78ee-87a9-4ad0-9c82-31de01389b34"
NOTE: Multiple crypto-volumes can be listed here, each resulting in a password prompt at boot time.
Build the new grub config:
grub2-mkconfig -o /boot/grub2/grub.cfg
NOTE: We should see entries for our old root as well, from the os-prober stage of grub2-mkconfig, though they may not work without some urging at the dracut prompt during boot.
This should produce a grub.cfg entry containing a kernel line reflecting our changes:
linux /vmlinuz-3.12.8-300.fc20.i686 root=/dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 ro quiet rd.luks.uuid=cc7e78ee-87a9-4ad0-9c82-31de01389b34
Finally, alter fstab to mount our new root volume:
/dev/mapper/luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 / ext3 defaults,user_xattr 1 1
NOTE: Over the years, distros have moved from mounting by device to mounting by label or UUID, or at least device-mapper name, since devices can change. We could do this here as well, for example mounting by the UUID of the filesystem within our crypto volume (UUID=551227f2-7ea4-4c5d-8bed-a5bff1d7354f). Fedora seems to mount crypto volumes by device-mapper name and so shall we.
Crypttab? It is possible to create an /etc/crypttab file containing an entry like this:
luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 /dev/mapper/Volume00-root none
A stock Fedora install on a crypto-root does not create this file, but its existence may (or may not) eliminate the need to specify rd.luks.uuid on the kernel command-line, among other things. I did not create this file on my system. Your mileage may vary.
Boot the New Root
At this point, we can boot into our new crypto-root. Following the above procedure should drastically reduce the chances of problems, but if they do occur, we are well-prepared to troubleshoot and recover with the knowledge we have gained and the backups we have made (see Working GRUB and Working Dracut).
We are left with this on the system drive:
/dev/sda (150GB)
sda1: ext3 /boot (300MB)
sda2: ext3 / (38GB)
sda3: lvm Volume00 (112GB)
Volume00-home: ext3 /home (70GB)
Volume00-swap: swap (2GB)
Volume00-root: LUKS (40GB)
luks-cc7e78ee-87a9-4ad0-9c82-31de01389b34 ext3 (40GB)
Ultimately, a successful move to a new crypto-root. In part three, we will concern ourselves with other volumes, namely home.
Permalink
06.16.14
Posted in Technology at 23:15:04 by Jeff Hardy
My ten year Fedora upgrade streak, now at 20, is still running on hardware that is nearly as old and I thought it time to move the install to a (slightly) newer laptop that I am using more frequently. This will involve a number of storage operations and system re-configuration, new concerns such as encryption, and more to come. Cloning an install and re-fashioning it in such a way will indeed be more work than a fresh install and copy, but imposes some interesting obstacles to work around, and remains within the spirit of maintaining a legacy install.
Here in part one, we concentrate on merely getting the system onto a new drive, and booting the new hardware. This will merge the data from two drives in its current desktop environment, to a single new laptop drive. What follows assumes that cloning, partitioning, and LVM operations are conducted with the old and new storage accessed from a Linux system in an offline manner. Single or rescue mode are not required as long as we can access the block devices, perhaps connected as secondary PATA/SATA or USB drives. In no case should they be mounted anywhere.
Dump the Drive Images
There are certainly more exotic tools for disk cloning, but let us stick to the good old UNIX standby ‘dd.’ We can be tricky and dd over a network pipe directly to a destination drive, but with storage available (in this case external USB mounted at /storage), we get a safe offline copy/backup of the original drives:
dd if=/dev/sda of=/storage/fritzdesk_sda
dd if=/dev/sdb of=/storage/fritzdesk_sdb
The partition info on sda, revealing our /boot and / partitions:
fdisk -l fritzdesk_sda
Disk fritzdesk_sda: 37.3 GiB, 40037760000 bytes, 78198750 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa5cba5cb
Device Boot Start End Blocks Id System
fritzdesk_sda1 * 63 619919 309928+ 83 Linux
fritzdesk_sda2 619920 78185519 38782800 83 Linux
The partition info on sdb, revealing our LVM partition containing logical volumes for home and swap:
fdisk -l fritzdesk_sdb
Disk fritzdesk_sdb: 74.5 GiB, 80026361856 bytes, 156301488 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x59d50c69
Device Boot Start End Blocks Id System
fritzdesk_sdb1 * 63 156296384 78148161 8e Linux LVM
Ultimately, we have this from our source system:
/dev/sda (40GB)
sda1: ext3 /boot (300MB)
sda2: ext3 / (38GB)
/dev/sdb (72GB)
sdb1: lvm Volume00 (72GB)
Volume00-home: ext3 /home (70GB)
Volume00-swap: swap (2GB)
Clone to the New Drive
We will combine the partitions from these two source drives into the new 150GB laptop drive. No matter how the new drive is connected (direct PATA/SATA or USB), we will assume the new drive is /dev/sdc for clarity.
First a clone of sda, as that contains our MBR, boot, and root:
dd if=/storage/fritzdesk_sda of=/dev/sdc
Check partitions on the new drive:
fdisk -l /dev/sdc
Disk /dev/sdc: 149.1 GiB, 160041885696 bytes, 312581808 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa5cba5cb
Device Boot Start End Blocks Id System
/dev/sdc1 * 63 619919 309928+ 83 Linux
/dev/sdc2 619920 78185519 38782800 83 Linux
We now need to create an LVM partition to hold the contents of sdb:
fdisk /dev/sdc
new, primary, partition number 3, first sector default, last sector default
type, partition number 3, 8e
write
Check partitions on the new drive:
fdisk -l /dev/sdc
Disk /dev/sdc: 149.1 GiB, 160041885696 bytes, 312581808 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa5cba5cb
Device Boot Start End Blocks Id System
/dev/sdc1 * 63 619919 309928+ 83 Linux
/dev/sdc2 619920 78185519 38782800 83 Linux
/dev/sdc3 78186496 312581807 117197656 8e Linux LVM
We are interested in the one LVM partition inside the fritzdesk_sdb image rather than the entire drive. We must expose it with kpartx:
kpartx -av /storage/fritzdesk_sdb
add map loop0p1 (253:3): 0 156296322 linear /dev/loop0 63
Scan for the new LVM physical volume just exposed to ensure everything was found:
pvscan
PV /dev/mapper/loop0p1 VG Volume00 lvm2 [74.52 GiB / 3.52 GiB free]
NOTE: It is good practice to ensure unique VG names when doing system installs, lest a generic name like Volume00 collide with another, such as when used in this way on another system. Luck is on our side.
Clone the exposed source LVM partition to the new LVM partition, physical volume to physical volume:
dd if=/dev/mapper/loop0p1 of=/dev/sdc3
Claim New LVM PV Space
This last clone was of a smaller LVM partition image into a much larger LVM partition on the new drive. We will want to claim that space for future LVM use. The pvresize command will claim all space up to the partition or block device boundary (now further out) unless otherwise specified:
pvresize /dev/sdc3
pvs
PV VG Fmt Attr PSize PFree
/dev/sdc3 Volume00 lvm2 a-- 111.76g 40.76g
vgs
VG #PV #LV #SN Attr VSize VFree
Volume00 1 2 0 wz--n- 111.76g 40.76g
NOTE: This may need to be prefaced with a pvscan to find the physical volume.
Now we have an additional 40.76GB in the LVM volume group to grow or create new logical volumes.
Boot the New Machine
At this point, we have directly cloned the system to a new drive with the fewest changes possible. With that drive physically installed as the sole drive in the new system, we boot to ensure all pieces are present, troubleshooting any startup issues and checking things before introducing any more complexity. For example, there is trouble starting X11, and a boot to single mode and check of the X11 logs indicates that there is a need to install xorg-x11-drv-intel. Other minor issues involve installation of xorg-x11-drv-synaptics for full touchpad functionality. Things of this nature are not surprising given a move to new hardware.
We are left with this on the destination drive (now /dev/sda in-use in the new system):
/dev/sda (150GB)
sda1: ext3 /boot (300MB)
sda2: ext3 / (38GB)
sda3: lvm Volume00 (112GB)
Volume00-home: ext3 /home (70GB)
Volume00-swap: swap (2GB)
unallocated: (40GB)
Ultimately, a simple, successful clone of fritzdesk to fritzlap. We have left some room to maneuver, which we will soon need. In part 2, we will worry about encryption.
Permalink
01.30.14
Posted in Technology at 10:03:36 by Jeff Hardy
First ensuring I was running fedup-0.8 to avoid the fedup-0.7 upgrade bug, this Fedora upgrade was off to the races with fedup --network 20. Unfortunately, after rebooting to the upgrade, my machine ended up wedged at the following:
systemd[1]: Failed to initialize SELinux context: No such file or directory
This fedup selinux bug is easily worked around with selinux=0 enforcing=0 on the kernel command-line. Afterwards, the upgrade completed without issue. I am also pleased to see that one particular Pidgin bug that plagued me throughout Fedora 19 seems to be fixed.
Meanwhile, an unrelated fedup upgrade I conducted on a laptop required that I work around some bugs related to finding my encrypted root partition, by specifying rd.luks.uuid=<uuid> in grub. I find myself wondering if we as an industry are making adequate progress reconciling and mitigating the increasing problems that come with increasing technical complexity.
Permalink
12.19.13
Posted in Technology at 22:16:49 by Jeff Hardy
SimpleSAMLphp is the ideal choice as a SAML 2.0 IdP for SUNY Federation. As it will be the core authentication mechanism for SUNY services, high-availability and load-balancing are an important consideration. Here is one approach to achieving this with HAProxy and Keepalived.
We intend to have a pair (at least) of Apache servers running identical SimpleSAMLphp installations. These will sit behind a pair of load-balancer machines running HAProxy and Keepalived, the first to balance HTTP traffic to the SAML servers, the second to provide virtual IP failover between the balancers. This forward-facing virtual IP will be stunnel-encryped, and advertised as our SAML Identity Provider (IdP).
__________vip__________
______keepalived______
balance1____balance2
|____\__haproxy__/____|
saml1___________saml2
Senior sysadmin Greg Kuchyt pioneered our use of these technologies for load-balancing and failover of our 389-DS LDAP services in a manner much like this, an approach that has served us well for many years, and which we have used for other services.
Our SAML setup is based on the document collection in the SUNY IDM Wiki, particularly the excellent Installing SimpleSAMLphp for SUNY Federation guide. Suffice it to say that I will not go into any detail on our IdP setup, instead covering only what SimpleSAMLphp changes are necessary to share session data for load-balancing and failover.
Shared Session Data with Memcache
On the SAML servers, in order to pave the way for balancing load across multiple machines, we need to ensure that SimpleSAMLphp session handling is switched over to memcache:
yum install memcached php-pecl-memcache
service memcached start
chkconfig memcached on
Note that SimpleSAMLphp uses PHP memcache and not the newer PHP memcached, but both use the system memcached daemon.
Firewalls will need to be opened to allow incoming memcache connections on port 11211.
In the SimpleSAMLphp configuration, set the store.type, and then adjust the memcache_store.servers array to taste (see docs). The following will cause all session data to be saved to the memcache servers on both boxes:
config.php:
'store.type' => 'memcache',
'memcache_store.servers' => array(
array(
array('hostname' => 'saml1.potsdam.edu'),
),
array(
array('hostname' => 'saml2.potsdam.edu'),
),
),
Load-Balancing with HAProxy
On the balance servers we first install HAProxy. The following configuration uses the least connection algorithm to balance HTTP sessions between the two SAML servers behind them.
/etc/haproxy/haproxy.cfg:
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
listen https
balance leastconn
bind 127.0.0.1:4430
option httpclose
option forwardfor
option httplog
option httpchk GET /
server saml1 saml1.potsdam.edu:80 check inter 5000 downinter 500
server saml2 saml2.potsdam.edu:80 check inter 5000 downinter 500
Note that hostnames are used here for clarity, but it is probably preferable to use IP addresses for server config lines. Also note that we bind our instance to localhost:4430, since we do not intend for public connections to hit this port directly. Instead they will hit SSL on port 443 provided by stunnel (later).
Start haproxy:
service haproxy start
We now have two balancers that will each balance HTTP connections to the SAML servers.
Failover with Keepalived
On the balance servers we next install Keepalived. On balance1, the following configuration establishes a virtual IP address to be maintained between the balance servers.
/etc/keepalived/keepalived.conf:
global_defs {
notification_email {
devnull@potsdam.edu
}
notification_email_from devnull@potsdam.edu
smtp_server 10.137.110.104
smtp_connect_timeout 30
}
vrrp_instance VI_1 {
virtual_router_id 1
state MASTER
priority 100
interface eth2
smtp_alert
authentication {
auth_type AH
auth_pass SomeKindofPasswordHere!
}
virtual_ipaddress {
10.137.100.101/24 brd 10.137.100.255 dev eth2
}
}
On balance2, we install the same configuration with two key differences:
state BACKUP
priority 50
Start keepalived:
service keepalived start
We now have a front-facing virtual IP address 10.137.100.101 providing failover between the balancers. You can view the status of the IP with: ip addr show.
SSL with Stunnel
On the balance servers we next install stunnel to encrypt communication on the virtual IP port 443.
/etc/stunnel/https.conf:
#CAfile = /etc/pki/tls/certs/entrust-chain.crt
cert = /etc/pki/tls/certs/saml.potsdam.edu.crt
key = /etc/pki/tls/private/saml.potsdam.edu.key
[https]
# public virtual ip address
accept = 10.137.100.101:443
connect = 127.0.0.1:4430
verify = 0
An example self-signed cert:
openssl genrsa -aes256 -out pass.key 2048
openssl rsa -in pass.key -out server.key
openssl req -new -key server.key -x509 -out server.crt -days 999
Note that key and cert will need to be copied into the locations referenced above.
Start stunnel with the following, probably in an init script:
stunnel /etc/stunnel/https.conf
With stunnel running, we now have encrypted communication on our virtual IP.
Hacking SimpleSAMLphp for Offloaded SSL
SimpleSAMLphp makes use of internal functions to determine port number, HTTP versus HTTPS, and to build links based on those determinations. Because we offloaded SSL to our balancer, it throws off these functions, with the result that links and such are improperly constructed.
Discussion at https://groups.google.com/forum/#!topic/simplesamlphp/m0aqJoURl7I suggests that there may be simpler ways to do this, eventually, but I settled on the changes represented in the following patch to lib/SimpleSAML/Utilities.php:
--- Utilities.dist.php 2013-04-08 04:44:05.000000000 -0400
+++ Utilities.php 2013-12-14 15:35:04.000000000 -0500
@@ -87,6 +87,8 @@
*/
public static function isHTTPS() {
+return TRUE;
+
$url = self::getBaseURL();
$end = strpos($url,'://');
@@ -105,6 +107,8 @@
*/
private static function getServerHTTPS() {
+return TRUE;
+
if(!array_key_exists('HTTPS', $_SERVER)) {
/* Not an https-request. */
return FALSE;
@@ -128,6 +132,8 @@
*/
private static function getServerPort() {
+return '';
+
if (isset($_SERVER["SERVER_PORT"])) {
$portnumber = $_SERVER["SERVER_PORT"];
} else {
After applying the patch, we should find that all clicks everywhere remain based on https://saml.potsdam.edu.
Permalink
« Previous entries Next Page » Next Page »