Well I've found you can avoid the extra sounds as long as its available on YouTube music specifically, as that is a music specific platform but you have to only select from the songs section and not the Videos section. You get less selections than you would've before but not really since those extra ones weren't even songs, but rather music videos.

Also Lyric videos basically never have any sounds in them.

(Doesn't address the quality issues since many YouTube videos are limited in audio quality, I've found downloaders that can do it but it's hit or miss).