OK guys, I finally found what the issue is, or at least kind of where it's coming from.
As some of you (and myself) suspected, my hot end is not reaching the reported temperature. I previously blamed the low readings on my IR thermometer on not being able to point the laser directly at the hotend, but it seems it was reporting accurate readings (around 95C when klipper reports 200C).
Now, here's where things get a little weird. At this point, I've used multiple thermistors, but swapped in a new one anyways. My board also has a pin for a second extruder thermistor, so I plugged it in to that one and changed the pin in my printer.cfg. No change.
I tried switching the bed and hot end thermistors on the board and in printer.cfg, no change.
I changed the thermistor "sensor type" from "EPCOS 100K B57560G104F" (same as the bed) to "Generic 3950", no change.
I found an article about tuning your pullup_resistance value. My cfg file did not have this value specified, so I added a line and started with the default of 4700, which made no difference (I'm assuming this value is loaded from the sensor type by default?). I toyed with the values until my thermometer read ~220C when setting the printer to that temp. However, to achieve this I had to adjust the pullup_resistance from 4700 to 13k+ (far beyond what should be needed) which makes klipper report 6C at room temp (print bed reports 27C). Unsurprisingly, I can hand-feed all the filament I want, but the temp reading is only now only accurate at 220C rather than only being accurate at room temp.
The thermistor, I feel, can be removed from the suspect list, as multiple thermistors exhibit identical properties.
I also feel the motherboard can be removed as well; there are three pins for thermistors, all three show accurate readings for the bed but identically inaccurate readings for the nozzle.
This only leaves software/ firmware, which I find incredibly odd for three reasons. For one, the printer was not even shut off in between "working" and "not working"; I successfully completed a print, and without shutting down, updating any configs, changing any settings etc., I swapped out the nozzle, and the printer hasn't worked since. Second, both the bed and nozzle thermistor are configured exactly the same, so if the nozzle is not set up properly the bed should be wrong too. Finally, Klipper is really straightforward and it's easy to configure things that commonly need configuring, it doesn't seem right that a configuration got changed and I'm completely incapable of finding what happened and fixing it.
As a Temporary Fix^TM^, I'm inclined to get a nice reliable probe thermometer, calibrate a pullup resistance value for common print temps, then updating my cfg whenever I want to change temps more than ~5c. This is obviously not even close to an ideal solution, but I don't know what else to try. Everyone else I've seen with this issue has resolved it either through hardware replacement or fixing settings, and I've tried all I can with both.
One word of caution re IR thermometers, the emissivity of the surface you're aiming at is going to massively impact its reading. From earlier photos it looked like you have a nickel plated block, most IR thermometers are preset to expect a 0.95 emmisivity (1 being a perfect black body), bare metals tend to be closer to 0 with polished nickel being 0.05, oxidised being in the middle 0.5 ish. Why I mention this is unless you are adjusting the thermometer for the expected emissivity it will totally underreport the surface temperature.
Other thing to consider is that surface temp and midblock temps can differ, forced convection from fans will totally contribute to that. Definitely like your idea of an accurate prove thermometer, I'd be super wary of changing your pullup resistance base on the ir reading, could definitely have it way too warm. Short term, you could try running with a higher temp setting. Have you probed the pullup resistor on the board?
Some thought that I wish I had considered earlier, sudden failure with no warning would definitely point me at electrical stuff. Might be worth considering swapping to something like an skr mini which is super accessible and is drop in replacement more or less for something like a Rambo board.
Yeah, I knew from the getgo that the IR thermometer was only gonna get me within a wide range of accuracy. (Hence resorting to more empirical testing like boiling water in the previous post, haha.) I was only using it initially to get an idea of where it was.
After posting yesterday I went out and got a high quality probe thermometer. I was actually lucky enough to get one who's probe is the same O.D. as a Bowden tube, so it friction fits inside my hot end. It took a little while to find the exact spot in the probe that reads the temp, but I was able to dial it in last night to where both Klipper and my probe read exactly 200c. (Before adjusting anything, Klipper would read 200c, IR would read ~90-130, probe reads ~170.)
I'm not familiar enough with electrical components/ readings to determine anything myself, but I had a buddy over yesterday that was helping me out who went to school for the stuff and is very well versed with 3d printer diagnostics, and he verified that nothing on the board is reading out of the ordinary. I considered a new board, but the board I'm using currently is itself relatively new (just had the manufacturer send me a new one after the original died; new board has been working for ~2 months).
Totally get not wanting to swap the board if it's that recent and been in serviced for that long. Super strange it's seemed to have drifted that much, could think of some last ditch things like changing the position of the thermistor, ensuring it's centred in the hole, set screw tightened enough but not enough to deform the sleeve but I know you've rebuilt and changed all the components a bunch of times, honestly could even just be your batch of thermistors, but it sounds like you have a way forward now.