So the spelling is irregular, so what. You’ll be bad at spelling for a while.

People mostly learn languages by reading.

having to memorize arbitrary gender for every noun in the language, learn complex verb conjugations, polite and impolite forms and make every verb and adjective agree with the nouns in gender and number

If you mess those up, people will still understand you. Saying "un chaise" instead of "une chaise" doesn't change the meaning and everyone knows what you're saying.

However, if you learn english words through text and then try to use them vocally, nobody will understand you. (looking at you "beard", who isn't pronounced at all like "bear" for some reason)

There is absolutely no correlation between spoken and written english, so in practice it's the same as having to learn two languages at once. Even adult native speakers still aren't sure how to pronounce simple 1 syllable words such as "route" or "vase", that's pretty telling how confusing that language is.