My standard is to test with very specific songs that have hard to hear nuances in them. If I can't hear those, the gear sucks. If I can, I'm happy. Don't care about anything else sound wise, and I have been able to ball on a budget while still maintaining that standard.

As you said, it's my ears that are the bottleneck, so I go with what works in that regard. There are shitty headphones that can't let me hear what I'm capable of hearing though.