Is the bad side of the seam where it stops or where it starts printing the outer wall? I assume it's where it stops and then it cross the wall to form the infill?

Correct, but this still happens when printing without infill, although less.

To add to the PA questions, are you sure that your PA setting actually are changing anything?

When printing the PA pattern, I can clearly see a difference. I haven't seen a huge difference in the actual print quality. It doesn't seem to have the any effect on this issue.

What printer is this and what firmware?

It's an anycubic kobra 2 bedslinger running klipper

What if you print the part significantly slower (to rule out rigidity/acceleration issues)

Tried printing at both 30mm/s and 100mm/s, it looks the same aside from slightly worse ringing at the higher speed.

It's an anycubic kobra 2 running klipper.

Randomized seams leaves the same mess, but randomly distributed around the model and not aligned like in the picture. But each seam is equally bad everywhere on the model.

Speed doesn't seem to affect the issue, I've tried both 30mm/s and 100mm/s

I disassembled the X and Y axes yesterday to check the motion system, moving by hand seems as smooth as it's ever been, no slip that I could feel. I think one roller on the X axis is very slightly oval unfortunately, but moving the model to different locations have no effect on this, and rotating models so seams align differently on the axes doesn't change anything, so I don't think that's the source of the issue.