You can switcj windows using keyvoard with apps like krunner, rofi, etc. Not exactly what you described/wanted.

I like rofi for this use case, but it uses fuzzy search instead of labels. You might have to type more than one letter, depending on what windows you have open. OTOH if you know any part of the window title you can start typing immediately without having to scan a list for a label first.

Labels work well for jumping to something you can already see, because the label appears where you are already looking, so you see it immediately. I'm guessing the process of finding the label for a window that is not visible would be clunkier - you'd have to find the label in a possibly long window list.