i simply don't understand how you square this concept of ideas existing outside language when that's inexpressable without language.

I think the easiest way to conceptualise it is when you're trying to explain something but struggling to find the right words - the idea is there fully formed in your mind, but you still have to search for the language to express it to someone else.