It's gonna be way less hassle to just use Linux. The gaming situation is so vastly improved from 6 or so years ago, and the vast majority of games just work, with a large amount of the rest only needing minor tweaks.

The big exceptions are in competitive gaming, and even there it's pretty much limited to proprietary & intrusive anti-cheats that I wouldn't have installed on my Windows computer anyway; Riot's Vanguard and FACEIT are probably the two big ones. Also Fortnite -- even though EasyAntiCheat does work fine with Linux, Epic has chosen to explicitly not support it. If you do play one of those few games -- or use other proprietary software like the Adobe suite that also won't work -- a dual boot should be fine, it only takes maybe two minutes to swap over and unless you have two beefy GPUs you'll be limited in a KVM setup.

About the second point, your pc shouldn't be putting any resources towards a VM if it isn't active. Just shut it down, close it, and play games on your host as normal.

One more thing, I'm assuming you're doing your gaming on Steam. Check ProtonDB for all of your games to see if they work on Linux (protondb only lists steam games, so if you have games as well, I've found a google search usually does the trick). You might find that you don't even need a VM

I used to do something similar. Passing GPU between host and VM without rebooting is a major pain in the ass. What I did instead was had a Linux hypervisor and 3 VMs (Linux, Windows, and MacOS). I would swap between the 3 VMs, and they each had access to my GPU. It was fun to set up and somewhat convenient, but got really annoying as it was my only workstation at the time.

I would highly suggest to just accept dual-booting and if it takes too long, get a faster SSD and/or faster RAM.

I've since gone Linux full-time, and I have no complaints. None of the games I can no longer play would be worth having Windows to deal with. I thought I would miss them at first, but I'm happy playing what's available.

I did this for a while with a 3080 and a a770. It works well, but after a while (and new Nvidia drivers) I switched fully to Linux.

• yes the GPU needs to be unused by Linux to allow for passthrough. There are some guides for making single GPU passthrough work but I never went down that road.

• see first response, making the Nvidia card available to both Linux and the VM is a hassle I never did. I would just game on Linux with the a770

• two separate PCs for the most part. I used udev rules to have Ctrl L + Ctrl R switch my input between devices

• pipe wire and udev rules. However audio is painful I actually used a USB physical switch to pass through things from host or VM. I would just pass through a USB controller to the VM and those USB ports on the back would be exclusively for the vm

I’ve played a small amount with it— I didn’t go far because I only have one graphics card. But my understanding is that you can set up the virtual machine to “capture” the graphics card when it is booted up, and “release” it when it’s done. That’s how some people can use a single graphics card, they just can’t see the Linux desktop while the vm is in use.

For switching games, make sure the drive the game is on is supported by both Windows and Linux. I imagine you’d need to unmount it on the host while the vm is open. I do this for some games with dual booting. Also, install the Windows version and force proton, otherwise you’ll have to install it twice.

For microphone, I assume there’s a solution like network: a way of forwarding the data to the VM in addition to being available on the host. I haven’t tinkered with this though.

For monitors, I’d look into Looking Glass. My understanding is that it can make a window to display the vm output. If you don’t mind the monitor being “tethered” to the gpu, though, I’d imagine any monitors attached on the gpu you pass would display the vm’s contents, while any on the host card will remain the same.

Best of luck!

I don't have a lot of input on this sadly but I would believe that the audio thing you mentioned could probably easily be fixed with pipewire.

As an example you can look at qpwgraph and see that you can pipe one input into several programs at once. I'm however unsure how KVM will handle it.