December 8

Linux: Create Network Bridge For KVM

The bridged network is a dedicated network interface to a virtual machine that helps virtual machines to connect outside the host machine.

Let us list the available network connections.COPY

nmcli connection show

Output:

NAME                UUID                                  TYPE      DEVICE
Wired connection 1  fbbdd6f9-0970-354e-8693-ff8050a85c77  ethernet  enp0s3

Now, we will create a virtual bridge network br0 with the help of physical interface enp0s3.

sudo nmcli con add ifname br0 type bridge con-name br0

sudo nmcli con add type bridge-slave ifname enp0s3 master br0

Next, we will assign the IP address of the physical interface to the bridge interface as the Bridge network interface will act as the primary network interface of your host system.COPY

sudo nmcli con mod br0 ipv4.addresses 192.168.0.10/24

sudo nmcli con mod br0 ipv4.gateway 192.168.0.1

sudo nmcli con mod br0 ipv4.dns "8.8.8.8","192.168.0.1"

sudo nmcli con mod br0 ipv4.method manual

KVM requires a few additional network settings. So, set them.COPY

sudo nmcli con modify br0 bridge.stp no

sudo nmcli con modify br0 bridge.forward-delay 0

Disable the physical interface and enable the network bridge.COPY

sudo nmcli con down "Wired connection 1" && sudo nmcli con up br0

Run the above command in the system terminal as you may lose SSH sessions when running them remotely.

Finally, check the network connections.COPY

sudo nmcli con show
Output:

NAME                 UUID                                  TYPE      DEVICE
br0                  ee117099-4935-4dde-a1f5-4981b0d9585e  bridge    br0
bridge-slave-enp0s3  492b5c81-e59d-4150-9b24-0348cd0dd87c  ethernet  enp0s3
Wired connection 1   fbbdd6f9-0970-354e-8693-ff8050a85c77  ethernet  --

By: Raj

Category: Linux | Comments Off on Linux: Create Network Bridge For KVM
December 3

Linux: Using fio for testing I/O performance

fio – The “flexible I/O tester”

fio is available on most distributions as a package with that name. It won’t be installed by default, you will need to get it. You can click apt://fio (Ubuntu) or appstream://fio (Plasma Discover) to install it (on some distributions, anyway).

fio is not at all strait-forward or easy to use. It requires quite a lot of parameters. The ones you want are:

  • --name to name your test-runs “job”. It’s required.
  • --eta-newline= forces a new line for every ‘t’ period. You’ll may want --eta-newline=5s
  • --filename= to specify a filename to write from.
  • --rw= specifies if you want to a read (--rw=read) or write (--rw=write) test
  • --size= decides how big of a test-file it should use. --size=2g may be a good choice. A file (specified with --filename=) this size will be created so you will need to have free space for it. Increasing to --size=20g or more may give a better real-world result for larger HDDs.
    • A small 200 MB file on a modern HDD won’t make the read/write heads move very far. A very big file will.
  • --io_size= specifies how much I/O fio will do. Settings it to --io_size=10g will make it do 10 GB worth of I/O even if the --size specifies a (much) smaller file.
  • --blocksize= specifies the block-size it will use, --blocksize=1024k may be a good choice.
  • --ioengine= specifies a I/O test method to use. There’s a lot to choose from. Run fio --enghelp for a long list. fio is a very versatile tool, whole books can and probably are written about it. libaio, as in --ioengine=libaio is a good choice and it is what we use in the examples below.
  • --fsync= tells fio to issue a fsync command which writes kernel cached pages to disk every number of blocks specified.
    • --fsync=1 is useful for testing random reads and writes.
    • --fsync=10000 can be used to test sequential reads and writes.
  • --iodepth= specifies a number of I/O units to keep in-flight.
  • --direct= specifies if direct I/O, which means O_DIRECT on Linux systems, should be used. You want --direct=1 to do disk performance testing.
  • --numjobs= specifies the number of jobs. One is enough for disk testing. Increasing this is useful if you want to test how a drive performs when many parallel jobs are running.
  • --runtime= makes fio terminate after a given amount of time. This overrides other values specifying how much data should be read or written. Setting --runtime=60 means that fio will exit and show results after 60 seconds even if it’s not done reading or writing all the specified data. One minute is typically enough to gather useful data.
  • --group_reporting makes fio group it’s reporting which makes the output easier to understand.

Put all the above together and we have some long commands for testing disk I/O in various ways.

Note: A file --filename= will be created with the specified --size= on the first run. This file will be created using random data due to the way some drives handle zeros. The file can be re-used in later runs if you specify the same filename and size each run.
Testing sequential read speed with very big blocks

fio --name TEST --eta-newline=5s --filename=temp.file --rw=read --size=2g --io_size=10g --blocksize=1024k --ioengine=libaio --fsync=10000 --iodepth=32 --direct=1 --numjobs=1 --runtime=60 --group_reporting

The resulting output will have a line under Run status group 0 (all jobs): which looks like:

  • WD Blue 500 GB SSD (WDC WDS500G2B0A-00SM50): bw=527MiB/s (552MB/s), 527MiB/s-527MiB/s (552MB/s-552MB/s), io=10.0GiB (10.7GB), run=19442-19442msec
  • The Seagate Momentus 5400.6: READ: bw=59.0MiB/s (62.9MB/s), 59.0MiB/s-59.0MiB/s (62.9MB/s-62.9MB/s), io=3630MiB (3806MB), run=60518-60518msec

The result should be close to what the hard drive manufacturer advertised and they won’t be that far off the guessimates hdparm provides with the -t option. Testing this on a two-drive RAID1 array will result in both drives being utilized:

  • Two Samsung SSDs: READ: bw=1037MiB/s (1087MB/s), 1037MiB/s-1037MiB/s (1087MB/s-1087MB/s), io=10.0GiB (10.7GB), run=9878-9878msec
Testing sequential write speed with very big blocks

fio --name TEST --eta-newline=5s --filename=temp.file --rw=write --size=2g --io_size=10g --blocksize=1024k --ioengine=libaio --fsync=10000 --iodepth=32 --direct=1 --numjobs=1 --runtime=60 --group_reporting

This will a line under “Run status group 0 (all jobs):” like

  • WRITE: bw=55.8MiB/s (58.5MB/s), 55.8MiB/s-55.8MiB/s (58.5MB/s-58.5MB/s), io=3378MiB (3542MB), run=60575-60575msec
Note: Many modern SSDs with TLC (Tripple Level Cell) NAND will have a potentially large SLC (Single Level Cell) area used to cache writes. The drives firmware moves that data to the TLC area when the drive is otherwise idle. Doing 10 GB of I/O to a 2 GB during 60 seconds – what the above example does – is not anywhere near enough to account for the SLC cache on such drives.You will probably not be copying 100 GB to a 240 GB SSD on a regular basis so that may have little to no practical significance. However, do know that if you do a test (assuming you have 80 GB free) of a WD Green SSD with 100 GB of I/O to a 80 GB file with a 5 minute (60*5=300) limit you’ll get a lot lower results than you get if you write 10 GB to a 2 GB file. To test yourself, tryfio --name TEST --eta-newline=5s --filename=temp.file --rw=write --size=60g --io_size=100g --blocksize=1024k --ioengine=libaio --fsync=10000 --iodepth=32 --direct=1 --numjobs=1 --runtime=300 --group_reportingYou need to increase size (files used for testing), io_size (amount of I/O done) and runtime (length the test is allowed to run to by-pass a drives caches.
Testing random 4K reads

Testing random reads is best done with a queue-depth of just one (--iodepth=1) and 32 concurrent jobs (--numjobs=32).

This will reflect real-world read performance.

fio --name TEST --eta-newline=5s --filename=temp.file --rw=randread --size=2g --io_size=10g --blocksize=4k --ioengine=libaio --fsync=1 --iodepth=1 --direct=1 --numjobs=32 --runtime=60 --group_reporting

Some example results:

  • The Seagate Momentus 5400.6: READ: bw=473KiB/s (484kB/s), 473KiB/s-473KiB/s (484kB/s-484kB/s), io=27.9MiB (29.2MB), run=60334-60334msec
  • WD Blue 500 GB SSD (WDC WDS500G2B0A-00SM50): READ: bw=284MiB/s (297MB/s), 284MiB/s-284MiB/s (297MB/s-297MB/s), io=16.6GiB (17.8GB), run=60001-60001msec

As these example results show: The difference between an older 5400 RPM HDD and a average low-end SSD is staggering when it comes to random I/O. There is a world of difference between half a megabyte and 284 megabytes per second.

Mixed random 4K read and write

The --rw option randrw tells fio to do both reads and writes. And again, a queue-depth of just one (--iodepth=1) and 32 concurrent jobs (--numjobs=32) will reflect high real-world load. This test will show the absolute worst I/O performance you can expect. Don’t be shocked if a HDD shows performance-numbers that are in the low percentages of what it’s specifications claim it can do.

fio --name TEST --eta-newline=5s --filename=temp.file --rw=randrw --size=2g --io_size=10g --blocksize=4k --ioengine=libaio --fsync=1 --iodepth=1 --direct=1 --numjobs=1 --runtime=60 --group_reporting

By: linuxreviews.org

Category: Linux | Comments Off on Linux: Using fio for testing I/O performance
December 3

Linux: Dual boot Windows/Ubuntu with secure LVM

The following is a very good tutorial it is not all inclusive, but very close:
  • Based on https://askubuntu.com/a/293029/286776
  • Installation date: 15-09-2018
  • Additional notes based on my own experience
  • The process describes a completely fresh installation with a complete repartitioning, however it should work fine when Windows is already installed (eg. brand new machine with Windows preinstalled).
  • The process was conducted on Dell’s XPS 15 9570 (2018) with specs:
    • CPU: i7-8750H
    • Screen: 4K with Touch
    • RAM: 16 GB (original) / 32 GB (manually upgraded)
    • Drive: 512 GB (SK Hynix PC401)
    • Windows 10 Pro license
    • BIOS version: 1.3.1
      • Suprisingly, Ubuntu’s update manager supports BIOS updates out of the box
  • My installation did not require to disable TPM nor Secure Boot
  • My installation did not force me to recover Bitlocker after Ubuntu’s installation
    • Some people report that it was needed in their case
    • See “Additional notes” for more info about GRUB & Booting into Windows

1. Preparation (using another computer with Ubuntu)

  1. Create Windows installation USB stick
    • Download .ISO file from Microsoft’s webpage
    • Create bootable USB using WoeUSB
      • Ubuntu has an option to “restore” ISO images using Disks utility, but it does not work correctly (Windows installer asks for additional drivers)
      • I also had to compile WoeUSB because of some weird bug in the default Ubuntu’s (PPA’s) supplied package that would not let me finish the installation process
  2. Create Ubuntu installation USB stick
    • Download .ISO file from Ubuntu’s webpage
    • Create bootable USB using “whatever”
  3. Go to BIOS (F2) and switch from SSD’s “RAID mode” to “AHCI mode”

2. Install Windows

  1. Insert newly created bootable USB and start the installation process
  2. Re-partition your drive
    • My partitioning scheme (devices might have be labeled differently!):
      1. Boot drive for Linux: 1GB
        • /dev/nvme0n1p1
      2. Windows OS drive: ~75GB
        • /dev/nvme0n1p5
        • Will automatically create additional drives before the actual OS drive as soon as you create the first “regular” partition
      3. Windows data drive: ~100GB
        • /dev/nvme0n1p6
      4. Ubuntu LUKS drive: ~300GB
        • /dev/nvme0n1p7
        • Can be created later
  3. Install Windows on the “Windows OS drive”
  4. Boot to Windows after installation, install all updates
  5. Enable BitLocker on “Windows data drive” (“Windows OS drive” was already encrypted)
  6. Create recovery data for both Bitlocker-protected drives and store them somewhere (eg. additional USB)

3. Install Ubuntu

  1. Insert newly created bootable USB and start the installation process
  2. Create LUKS container on “Ubuntu LUKS drive” and “wipe it”:
    • sudo cryptsetup luksFormat /dev/nvme0n1p7
    • sudo cryptsetup luksOpen /dev/nvme0n1p7 cryptdrive
    • sudo dd if=/dev/zero of=/dev/mapper/cryptdrive bs=16M
  3. Create LVM physical volume, a volume group & logical volumes:
    • Volumes are sized as follows:
      • OS drive: 60GB
      • Swap: 16GB
      • Home: rest
    • Commands:
      • sudo pvcreate /dev/mapper/cryptdrive
      • sudo vgcreate vglinux /dev/mapper/cryptdrive
      • sudo lvcreate -n root -L 60g vglinux
      • sudo lvcreate -n swap -L 16g vglinux
      • sudo lvcreate -n home -l 100%FREE vglinux
  4. Start the installation process using GUI:
    • Connect to WiFi network
    • Installation options:
      • Pick “minimal installation”
      • Pick “install updates during installation”
      • Pick “install 3rd party drivers” (no need to “reset Secure Boot password”)
    • Pick “custom installation process” (the one which lets you repartition stuff)
      • Pick /dev/mapper/vglinux-root as ext4 FS & mount it to /
      • Pick /dev/mapper/vglinux-home as ext4 FS & mount it to /home
      • Pick /dev/mapper/vglinux-swap as swap
      • Pick /dev/nvm0n1p1 as ext4 FS & mount it to /boot
        • Without doing this, installation will fail when configuring GRUB
      • Pick “boot drive” (the select list at the bottom) and assign it to /dev/nvm0n1p1
    • Proceed with the installation
  5. After GUI installation completes, stay within the Live USB environment
  6. Check the UUID of the LUKS drive:
    • sudo blkid /dev/nvme0n1p7
    • Example output:
      • /dev/nvme0n1p7: UUID="abcdefgh-1234-5678-9012-abcdefghijklm" TYPE="crypto_LUKS"
  7. Mount root & boot drives and chroot into the main mount:
    • sudo mount /dev/mapper/vglinux-root /mnt
    • sudo mount /dev/nvme0n1p1 /mnt/boot
    • sudo mount --bind /dev /mnt/dev
    • sudo chroot /mnt
    • mount -t proc proc /proc
    • mount -t sysfs sys /sys
    • mount -t devpts devpts /dev/pts
  8. In chroot env, configure crypttab allowing to boot Ubuntu with Encryption unlocker
    • sudo nano /etc/crypttab:# <target name> <source device> <key file> <options> # options used: # luks - specifies that this is a LUKS encrypted device # tries=0 - allows to re-enter password unlimited number of times # discard - allows SSD TRIM command, WARNING: potential security risk (more: "man crypttab") # loud - display all warnings cryptdrive UUID=abcdefgh-1234-5678-9012-abcdefghijklm none luks,tries=0,discard,loud
    • update-initramfs -k all -c
  9. Reboot into Ubuntu

4. Ubuntu Tweaks

  1. Run some of the dell-xps-9570-ubuntu-respin tweaks:
    • Main script is here: https://github.com/JackHack96/dell-xps-9570-ubuntu-respin/blob/master/xps-tweaks.sh
    • Modify the GRUB editting code to include splash option
    • Nvidia & WiFi card changes were not performed
    • Running this script yielded power consumption reduction from ~16W to ~7W (much closer to Windows) in Idle
  2. Additional packages (some might have been included in the Respin):
    • sudo apt-get install smbios-utils
  3. More Tips & Tricks related to Thermals:
    • Some useful tips and tricks: https://github.com/JackHack96/dell-xps-9570-ubuntu-respin/wiki/Some-useful-tips-and-tricks
    • Switch to Battery-Mode on AC for better Thermals:
      sudo tlp bat
  4. Enable right-mouse-click:
    • Follow these instructions: https://askubuntu.com/a/1029458/286776
  5. Enable Yaru-dark in GNOME Shell:
    • Follow these instructions: https://www.linuxuprising.com/2019/10/how-to-get-dark-gnome-shell-menus-and.html
  • Latest commit used: 7486bc9b3bbf3aca26bd8ab48b39788a7f9392aa
  • Nvidia Drivers used: 390

Additional notes

  • Switching between OSes:
    • Ubuntu (GRUB) is the default boot option
    • Windows can be accessed using F12’s boot option menu
      • It can also be accessed using GRUB’s menu, however, then it prompts me to use the BitLocker’s recovery key
      • Cancelling the recovery and using F12 -> Windows Boot Manager trick did not prompt to use recovery key again…
  • Both, when installing Ubuntu, and sometimes when rebooting installed Ubuntu, “poweroff” or “reboot” results in prolonged shutdown with locked up display.
    • Fixed with dell-xps-9570-ubuntu-respin tweak script
  • About BIOS upgrade:
    • An upgrade from 1.3.0 to 1.3.1 required to swap RAM sticks to the original ones again, because the machine would not boot with white & amber LED flashes (supposedly indicating “memory problem”). After booting just once with the original sticks, I’ve swapped to the 2x16GB sticks again without a problem.

Changelog

  • [2019.10.24]
    • Added link to an article related to enabling Yaru-dark in GNOME Shell (eg. notification center background adjustment, which by default is white).

By: M Dziekon

Category: Linux | Comments Off on Linux: Dual boot Windows/Ubuntu with secure LVM
December 2

Linux: Gentoo rename an interface

If you accidentally set the network up with the wrong name you can do the following:


Senario:
When setting up Gentoo called the network net.enps03
Network device is actually net.enp0s3

Using rc

rc-service delete net.enps03
mv /etc/init.d/net.enps03 /etc/init.d/net.enp0s3
rc-service add net.enp0s3 default

rc-service net.enp0s3 start

Category: Linux | Comments Off on Linux: Gentoo rename an interface