By default virt-manager with libvirtd+qemu will give you the option to pass your Virtual Machine’s audio to the host’s pulseaudio daemon. This causes some problems as you have limited options to change the value/device while inside the Virtual Machine and it can be buggy or not work at all as was my case in Ubuntu 21.04.

Use PCI Passthrough to take over your system’s sound card and usb controller

To solve this problem I use VFIO’s PCI Passthrough to take the system’s sound card and usb controller away from the host operating system and give it to the Virtual Machine. This automatically handles my keyboard, mouse, and audio devices and removed various latency problems I was facing.