Can't think of the most difficult problem, but I have managed to solve a lot of problems with btrfs snapshots.

I used a GPS module to feed time data to a custom ntp service with no jitter. About as close as I could get to atomic clock precision for under $150.

single gpu vm passthrough. took a few days for troubleshooting, and i didnt even want to get it to be undetectable by game anticheat, i hear that needs building your own kernel for some advanced detection methods.

I've generally had good luck with hardware and things just worked under linux. But one day I upgraded a few machines on my network to 2.5G ethernet. Several already had the ports, but my little NUC NAS box didn't, so I installed a 2.5G usb ethernet dongle. No matter what I did, I couldn't get it to work. It would show up and NM would act like it was up and there were no errors or anything, but it just wouldn't actually function.

Eventually, I found out that it has a built in USB data partition that contains the drivers for windows. The card was coming up as a usb disk first when the hardware was assigned and not a network card which it should have been.

I had to write a blacklist the usb modules first, which I had done before, but I had to also write a udev rule to automatically add the network card and driver on boot. It wasn't that difficult to actually do, but I had just never had to do anything with udev rules before. Took me a good three days of troubleshooting to finally get everything to work correctly on boot.

ACTION=="add", ATTRS{idVendor}=="20f4", ATTRS{idProduct}=="e02c", RUN+="/sbin/modprobe r8152" RUN+="/bin/sh -c 'echo 20f4 e02c > /sys/bus/usb/drivers/r8152/new_id'"

Getting VR to work

while playing around with face/fingerprint unlock for my laptop, I messed up pam (Linux Pluggable Authentication Modules) and no passwords were working anymore except for the root account. At first I was still on my account, but then I stupidly rebooted and could only log in as root. After so many config edits, I gave up and instead booted up windows (my laptop's dual booted), setting up a new linux install in VirtualBox, and then copying over the PAM config files from the vm to the actual Linux install.

and it all somehow worked!

I am now facing another issue which I'm gonna say here in the hope somebody has already ran into it: after updating to KDE plasma 6, tap to click works on my touchpad, but actually, physically, pressing on the trackpad doesn't work. I can hear the pad's physical clicking noise, but nothing happens os wise

this one's still to be resolved

I have had an issue for years that I couldn't pinpoint to a root cause (I'm strongly inclined to think it's a kernel issue). I bought a CM Storm Quickfire TK keyboard with ABNT2 layout.

The issue is: every time I try to type any key that is not a letter or number one, the computer freezes for a full ten seconds before acknowledge the press and showing the character. Tried a bunch of Linux distros through the years and the issue persists. On Windows it works flawlessly.

Just give up trying to debug the problem, but I still have this hole in my heart where the cause of this issue lives.

At one point, my laptop's Nvidia drivers were all tangled up. The package dependency graph had portions of the screwy, we-don't-need-your-stinking-standard-version-scheme, binary blob drivers both in front of and behind the currently installed version. I had to basically gut everything Nvidia related, by performing surgery on the filesystem and Apt database, and then build it back. At one point, I was flying in text mode only; not hard, but worth mentioning since it shows how deep a cut this was.

Related: getting the above nonsense to cooperate with containers that also want to do GPU things. As much as I wanted this work with coding up a one-and-done solution (e.g. docker-compose or BASH script), you can't get away with mounting the host Nvidia driver and tools via volumes. The software on the container image itself must be built against the specific version you're running - no exceptions. So, I now rebuild these containers after every Nvidia package upgrade (from the author's git repo), which is a stupid way to achieve containerization. If Nvidia had a stable API/ABI across releases, this would just work. /rant

windows update kept downloading these bloated "updates" that included brand new software that I didn't want or use, broke my settings, added a bunch of spyware, adware and other shit and slowed down my system

installing linux fixed that instantly and permanently

A Gentoo upgrade package list with over 100 packages and conflicts all over the place. Then do it again when the list grows to the same size in a few months.

This is why I don't use Gentoo anymore.

Some programs still relying on python2 when the operating system has long since upgraded to python3.

Not really an issue per se, I just had to switch those apps over to using the flatpak version which would have it installed as needed. (I'm looking at you GIMP)

What's the deal with Linux and Tomoko? lmao

I keep seeing memes featuring her on Linux communities. Is it only the shut-in nerdy vibes?

I can't remember the details anymore, but for a year or two I had a bad run of absolutely hosing my boot config and leaving myself in a state where the system either couldn't find it's kernel or couldn't find the root partition and would drop me into an initramfs emergency shell. I got pretty good at booting into a live environment, getting all my dm-raid and lvm disks discovered, mounting all the relevant file systems in the right place, chrooting in and rebuilding the pieces that were broken

I installed nixos on my laptop. That was four weeks ago. I am still configuring it.

Xfree86 was sonetimes a mess. And i did not have a browser anymore when it refused to start. So man pages only.

I once rm -rf all the db files of a running database: Recovered the files via inodes since they were all still open on the running database, that was a mess.

I screwed up permissions on an LXC container in Proxmox by converting it from unprivileged to privileged (against recommendations) and had to mount it offline and write a script with find into chown via the execute flag to change all the UIDs and GIDs from the shifted unprivileged ones to the standard host-level ones.

Luckily this was in my own lab so it was a (mostly) harmless learning experience.

Biggest thing I noticed after switching is forum posts. In Linux ones you usually get a fix where the Windows ones 9/10 they just advise you reformat

That not a thing on Linux?

SFC /Scannow on Windows

Installing a hadoop cluster across 5 machines. I wouldn't say I fixed it, but I made it so it wouldn't collapse until long after I'd left that company.

idk how i would define difficult, but the thing i probably put the most time into figuring out thus far is LXC containers.

Or LXC, if you like not using redundant acronyms. Those containers are good shit, weird shit, but good shit nonetheless.

I had to fix so many booting problems using live usb, grub, xorg, login managers, they'are all difficult

At some point I've installed rust implementation of the coreutils from the AUR, they worked for a long while until some ssl vulnerability were discovered and everyone had to update the library. As you can imagine, without working coreutils system were hard to use. troubleshooting were also a pain in the ass because who could blame coreutils of all things? :P