You could either have socks already in pairs at drying time (we hangdry so we do this, just hang them together and when taking off, fold one into the other, they will not separate accidentally). Alternatively you could have all the same socks and not care.

Alternatively, you can just not care if yours socks match. I only care for my business socks because a) they all have silly designs and b) My line of work calls for a slight bit of professionalism in appearance, so I try to style my hair, clean up my facial hair and at least have matching socks, goofy as they might be. Thank god I don’t have to wear a suit.