Scope

This blog is a practice search about windows virtio driver installation.

Backgroud

For virtualization software, normally guest will install virtio related drivers to get better virtualization performance. But instll virtio driver to windows guest sometimes became complex, so many softwares offer a practice guide about virtio dirver installation

the guide introduce that how to install virtio driver from win-virtio.iso while lauch windows install.

But in practice, if user want install driver to exists guest, windows still get BSOD after virtio driver installed.

So I write this blog to solve related problems.

Newly virtio driver installation

Windows running root disk attached to ide controller and install virtio driver. Then stop guest and move the disk from ide controller to virtio-serial controller, start guest will meet BSOD (no accessible boot device).

This is because windows do not load the virtio controller when install virtio driver to running vm.

According to P2V practice, how to inject virtio driver to a guest https://portal.nutanix.com/page/documents/kbs/details?targetId=kA00e000000kAWeCAM

we need manually load driver

1

drvload vioser.inf

then install the driver to disk where windows installed:

1

dism /image:c:\ /add-driver /driver:vioscsi.inf

but if you try to do this on a running windows vm, dism will tell you that this operation is not allowed on a running windows. So the kv tell user to do the operation through cmd prompt when windows failed to boot not convinence if there are many guest need do this.

from superuser https://superuser.com/questions/1057959/windows-10-in-kvm-change-boot-disk-to-virtio/1253728#1253728

other solution is raised, the best one is by setting guest into safeboot mode

1

bcdedit /set "{current}" safeboot minimal

windows will load all drivers then change the disk controller seems make sense, but still manually operation is required.

A tricky way is noticed by adding a dummy virtio disk to windows and then install virtio driver, the controller will be loaded at first.

the following steps I followed:

1. Install the virtio driver in windows
2. Add a additional “dummy” virtio disk. Reboot and check if the “dummy” works.
3. If Step 2 works, then switch the boot disk to virtio.
4. Reboot
5. Remove the additional “dummy” virtio disk

because we do not need to do more operation inside guest, so this solution can be changed to a automatic way.

And more discussion can be found on reddit:

Looks like you’re having the issue of windows refusing to load the virtio storage drivers at boot.

The only thing I found that works for me is using this method - https://superuser.com/a/1200899. You can also try this method of adding another disk and installing the driver but I personally found that to be very hit and miss.

For the first method you need to use diskpart to assign drive letters to your windows drive and virtio iso this tutorial should help if you don’t know how to do it.

but luckily,

You need to install the virtio drivers on a per storage device basis.

I suggest swapping back to sata and add a empty virtio device to your guest. Then boot and install the virtio driver for the new the device. Last step is to delete the old sata device and mount the device image at the virtio device and boot your guest.

Make sure that libvirt didn’t changed the pcie address of your virtio device as windows registers the driver on a per device basis.

the dummy disk work around can be used because the pci address acutally reused (as the virtio device will be removed and reboot)

This method works well when virtio drivers are newly added, but if you have booted guest with virtio driver installed, change the controller from ide to virtio is complex.

We prefer user to install virtio driver during first windows intallation and make it as a image to avoid controller change.

Virtio driver already installed

While virtio already installed and reboot windows and the disk controller not changed. After reboot the boot disk is still ide.

If you attach virtio-blk disk to guest, it will be recognized and loaded right now.

But if follow the steps below to attach a dummy disk (in this case you attached a virtio-blk disk actually), change the ide controller to virtio will not work, windows kept report BSOD after changed.

Work around is uninstall the virtio driver and reinstall with the steps than reboot every will works.

I think maybe windows only load all drivers which are newly installed. But for existing driver, it only works per disk basis.

Virtio-scsi always works

Cheerfully, if you change the ide/sata controller to virtio-scsi controller after virtio driver installed, windows works well.

More performance test is needed because we kept use virtio-blk for root disk due to some version of virtio driver offered virtio-scsi has bad performance.

Live migration is a important part of kvm virtualization at the first day it was designed. However when dive into control plane of libvirt live migration, it became quite complex. So I will describe the basic implementation about it at the early stage.

Libvirt + QEMU basic building blocks

For KVM based virtualization software, normally use libvirt + QEMU to manage guest’s lifecycle. And for live migration we have to know some basic part between libvirt and QEMU.

the figure below introduces the basic parts and I just list those parts from left to right:

• virsh: a commandline interface to management domains
• libvirt sdk: Python, Go… supported sdk to access libvirt by defined api
• Libvirt api: exposed connect (the connection to libvirt), domain (guest), network (virtualization network of a hypervisor), storage volume (storage volume as block device which can be used by domain), storage pool (logically used for allocate and store storage volumes)
• QEMU driver: libvirt driver of qemu, it will translate libvirt api invoke to related qemu operations
• QEMU: a generic and open source machine emulator and virtualizer
• qmp: QEMU machine protocol, is a JSON-based protocol, which allows applications to control a QEMU instance

So when we do a live migration operation all those parts will be involved.

Libvirt live migration

For the control plane (libvirt), many concepts need to be introduced before we try to comprehensive its migration logic.

According to https://libvirt.org/migration.html there are two options for network data transport.

• Native transport: use qemu socket to transport data
  • Require network between hypervisor (firewall issue should be solved)
  • Encryption support is depend on hypervisor
  • Better performance (minimising the number of data copies)
• Tunnelled transport: the data will be transported through libvirt RPC protocol
  • Encryption supported
  • Less firewall issues
  • Worst performance (due to encryption)

And libvirt also support different control plane, the migration support have common features

1. a peer2peer flag decide if we use client to connect to libvirtd servers or libvirtd server manage the connection itself
2. A destination URI with a form like ‘qemu+ssh://desthost/system’ for libvirtd connection
3. Data transport URI need a optional URI like ‘tcp://10.0.0.1/‘ means use TCP for data transport to hypervisor or libvirtd server
4. Normally libvirtd on target will automatically determine its native hypervisor URI so is not required in migratin api
5. If hypervisor do not offer encryption itself, tunnelled migration should be used
6. When libvirt daemon can not access network use unix migration
7. For vm with disks on non-shared storage, remember copy all storages

Following are libvirt supported migrations and all available for qemu driver:

• Native migration, client to two libvirtd servers
• Native migration, client to and peer2peer between two libvirtd servers
• Tunnelled migration, client and peer2peer between two libvirtd servers
• Native migration, client to one libvirtd server
• Native migration, peer2peer between two libvirtd servers
• Tunnelled migration, peer2peer between two libvirtd servers
• Migration using UNIX sockets
• Migration of VMs using non-shared images for disks

Libvirt keepalive of client

Libvirt use a C/S architecture and during migration libvirt need to support ‘client to two libvirtd servers’ or ‘client to and peer2peer between two libvirtd servers’.

So connection management between client and server or server and server is important for libvirt. And some common conserns for this architecture:

• Client and server connection
  • Async task not relay on the connection if server implement idempotency
    • Domain object lock help with idempotency
  • Sync task relay on the connection
    • All sync tasks should fail if connection keepalive timeout
• Server and server connection
  • Source server should be treated as client and same with client and server connection

In order to solve basic requirements, libvirt introduced keepalive for client connection. Client can set a keepalive timeout with interval and count (server should support this because a keepalive response is required).

Note:

• Default settings is configured from libvirtd.conf
• If set keepalive timeout to 0 means disable keepalive for client

The code from src/rpc/virkeepalive.h is quite easy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

virKeepAlivePtr virKeepAliveNew(int interval,
                                unsigned int count,
                                void *client,
                                virKeepAliveSendFunc sendCB,
                                virKeepAliveDeadFunc deadCB,
                                virKeepAliveFreeFunc freeCB)
                                ATTRIBUTE_NONNULL(3) ATTRIBUTE_NONNULL(4)
                                ATTRIBUTE_NONNULL(5) ATTRIBUTE_NONNULL(6);

int virKeepAliveStart(virKeepAlivePtr ka,
                      int interval,
                      unsigned int count);
void virKeepAliveStop(virKeepAlivePtr ka);

int virKeepAliveTimeout(virKeepAlivePtr ka);
bool virKeepAliveTrigger(virKeepAlivePtr ka,
                         virNetMessagePtr *msg);
bool virKeepAliveCheckMessage(virKeepAlivePtr ka,
                              virNetMessagePtr msg,
                              virNetMessagePtr *response);

For libvirt keepalive timeout issue

The result from https://bugzilla.redhat.com/show_bug.cgi?id=1367620 butzilla explains a issue of live migration failure due to poor network and the connection between libvirtd servers down which will report a keepalive timeout error.

In libvirtd log (https://libvirt.org/kbase/debuglogs.html) we could see:

1
2
3
4
5

2023-01-05 05:07:36.721+0000: 114785: info : virKeepAliveTimerInternal:131 : RPC_KEEPALIVE_TIMEOUT: ka=0x7f6af4006c60 client=0x7f6af400
6a70 countToDeath=0 idle=30
2023-01-05 05:07:36.721+0000: 114785: debug : virKeepAliveTimerInternal:136 : No response from client 0x7f6af4006a70 after 5 keepalive
messages in 30 seconds
2023-01-05 05:07:36.721+0000: 114785: error : virKeepAliveTimerInternal:138 : internal error: connection closed due to keepalive timeout

And search for client=0x7f6af400 we can find it is a connection created during migration:

the dconn is the URI to destination libvirtd server.

For peer2peer live migration, this issue can be workaround by using seperate network for libvirtd connection and data transport.

First find download command from official page: https://www.tenable.com/downloads/nessus?loginAttempted=true and I get 10.4.1 rpm with curl command:

1
2
3

curl --request GET \
  --url 'https://www.tenable.com/downloads/api/v2/pages/nessus/files/Nessus-10.4.1-es7.x86_64.rpm' \
  --output 'Nessus-10.4.1-es7.x86_64.rpm'

and before installation need to disable firewalld and selinux:

1
2
3
4
5
6
7
8
9

systemctl stop firewalld

systemctl disable firewalld

setenforce 0

#set SELINUX=disabled in below file

vim /etc/sysconfig/selinux

then install the rpm:

1

yum localinstall Nessus-10.4.1-es7.x86_64.rpm

and start the service:

1
2
3

systemctl start nessusd

systemctl enable nessusd  #Gives error

access nessus through 8443

before active access:

https://www.tenable.com/products/nessus/nessus-essentials to get a activate code and go ahead for your trial of Nessus.