Stolen from a Reddit thread cause the top answer isn’t super accurate (tldr Japanese “nipon” to Portuguese to Italian to English)

The first three Europeans that arrived in Japan in 1543 were Portuguese traders (António Mota, Francisco Zeimoto and António Peixoto). They were on a Chinese trading ship that had been blown off course and stopped on the island of Tanegashima to take on fresh water.

The Portuguese had three names for Japan. This is evidenced by the title of the 1603 Portuguese Japanese dictionary (Vocabvlario da Lingoa de Iapam) which uses Iapam and within in its pages also provides two other pronunciations for Japan being iippon and nifon. The reason for the multiple names appears to be due to:

The Portuguese first got the name Japan from the Chinese which called it Riben in Mandarin. Iippon is a relatively close translation of this word that sort of works for the Portuguese tongue. However, the Chinese language of wayfarers and the one that the first Portuguese to arrive in Japan would have heard would have been either Shanghainese or Hokkien (the dialect from Fujian). Shanghainese would have pronounced Nippon as Zeppen. Hokkien would have pronounced Nippon as Ji̍tpún. Nifon would have been relatively close to both. The Japanese that the Jesuits, who compiled the dictionary, would have likely to have spoken would have been influenced by the Japanese spoken in Nagasaki, which is where the Portuguese main base was. The accent of Nagasaki is what is called a Nikei-accent system, and widely used in south-west Kyushu. It has two contrasting tonal patterns, irrespective of the number of moras in the word. Thus Nippon would be Ni-Pon which then translates to Ia-pam

The Italians then started using the term Iapam. The largest Italian city of that era was Padua. Given the round about way the word Iapam got to Padua and based on the Italian spoken then, it got translated to Giapan. In an English travel book published in 1577 called “The History of trauayle in the VVest and East Indies …” the term Giapan was used.

Given that the Italian Gi sounds like J, it is not surprising that the English swapped Gi for J resulting in Japan.

Thus how Nippon became Japan appears tortuous starting with Portuguese being influenced by the type of Japanese spoken by the Jesuits in the 16th Century. Then from the three terms that the Portuguese used, the one that was perceived and recieved in Padua was Iapam, which was then translated into Italian as Giapan. And then how Giapan, used in the first known English travel journal that used the name, became anglicized into Japan.

we call it Japan from the name Marco Polo gave it, Cipango, which should come from the name of some random island he ended up in. we call it Greece because the Romans called it Graecia, for reasons I can't figure out

There are actually specific words in the English language for precisely what you're talking about.

Endonym - The native name of a geologic feature or place. From your example: Nihon in Japanese.

Exonym - The foreign name of a geologic feature or place. From your example: Japan in English.

The short answer is that the name for "old world" countries in a language isn't translated, it is simply what "we" call "them", not what "they" call themselves.

Using Greece as an example for English, English has a lot of French influence, which in turn had a lot of Latin influence. It is believed the early Latin (ie modern day Italian) peoples first met Graecians, a tribe likely from Boeotia in modern day Greece, and used the name to refer to all people from the same place. By the time of the Romans, this was the name and was then spread throughout the empire, including back to Greece itself.

A more modern or current example would be how people often called The Netherlands, Holland. Same idea, just several millenia apart.

I wish r/askhistorians mods would come here. This is a great question for them, if you’re looking for in depth and cited replies.

There is also a other example in Europe, Finland. We finns call it Suomi, and everyone else calls us Finland. Origin of Finland is also from Romans, because they called us Fenni. Origin of Suomi is unknown, but there are multiple theories.

It’s strange that we still use these weird synonyms in English for real names that are just as easy to pronounce.

And how did we get Finnish from Suomi?

Edit: I missed the other comment with the answer. Once again it was Romans

Another possibility people here aren't mentioning: many countries are new.

For example, Greece. There's talk about "Ancient Greece", "Hellenistic Greece", "Classical Greece", etc. But, in 227 BC Sparta invaded Achaea. In the 6th century BC, Ionia was "Greek" but is on what is now part of Turkey. In the 1200s there was the Principality of Achaea, the Dutchy of Athens, the Kingdom of Thessalonica (under Constantinople) and Negroponte, Rhodes, Crete, etc. which were under Venice.

The idea that there are a group of people who consider themselves part of a whole that includes Athens, Rhodes and Kavala but not Constantinople or Izmit (Nicomedia) is pretty new. Since the name "Greece" is older than the modern idea of "Greece", it's no surprise that there's a mismatch in the names.

If you look at the characters the Japanese use for the name of their country, 日本 is pronounced Nihon in Japanese. The same characters in Chinese spell Riben (in Pinyin) which is pronounced closer to "Erben" but almost with a J sound, especially in the northern part of the country. If you heard the Chinese pronunciation, you could easily understand how a foreigner would translate it to "Japan."

When people speaking one language encounter another people, then the first group will only refer to the second group by their endonym if they respect the second group's autonomy and identity. Their first instinct is just to call them by whatever details might distinguish them ("Netherlands" because the land is pretty close to sea level), or even by what they might say often (the somewhat derogatory "gook" comes from when American soldiers heard Koreans say "Miguk" which is "America" in Korean; also, I think there are some Indigenous Americans whose English names were simply their language's translation of "I don't understand what you are saying", but I can't remember what groups).