It was a long gradual process that didn't start with isekai; it has been decades of gradually accelerated industrial pressure to pander to the biggest spending otakus in Japan (and weebs in the west) because they bought the merch.

The pandering used to be pantyshots during an otherwise passable fantasy adventure focused on the adventure. Over time the focus increasingly became pandering to the audience, not just in horny/creepy stuff but also in "your video game skillz allow you to metagame and conquer this fantasy world" powerwanks.

The escapism and power fantasy intrinsic to the genre combined with modern anime being designed to appeal to the type of otaku who'll spend hundreds on merch or dvds (or $40k on a statue of their waifu). For these types there's an appeal in a girl who literally can't leave them, hence slavery. Harems are partly wish-fulfilment, partially marketing: 5 archetypal waifus will appeal to more otaku than 1 archetypal waifu.

We started off with Magic Knight Rayearth, Escaflowne and Fushigi Yuugi too how did everything go so wrong (answer is capitalism, duh)

Ok Fushigi Yuugi could be pretty problematic at times but still!

There are a few proto examples from western Baentrash fantasy like The Radiant Knight (do not read) that got picked up where the power fantasy (that was already being deconstructed in novels like A wizard in spite of himself and The man who came too early.) was mixed with dumb horniness.

As the chan culture picked up and became more right wing and misogynistic both here and in Japan, this went from moderately problematic to a death spiral dropping all the cool social history and cultural/psychological examination parts of portal fantasy in favour of having a little world that you get to do free war crimes to but it's good actually.

Nyaa link (torrent)
9anime link (stream)
Stremio is my preferred option personally, best of both
apparently there's only 3 episodes out so far, so you might wanna wait

Animepahe is our unofficial favorite anime piracy place here. Unless stuff has changed I didn't hear about

I believe it's currently airing so it'll be one you have to pick up weekly if you do.

"Normal world" knowledge often does play a part, but more than that I think part of the draw of isekai stories is that the audience can more easily empathize with a character originally from our world trying to make sense of the fantasy they find themselves in than characters who are from the fantasy world and thus don't find the setting particularly special.

It's not like straight fantasy stories have died out, though. In the current anime season alone you have Sousou no Frieren, Goblin Slayer, S-Rank Musume, and Mahoutsukai no Yome—and no doubt several more—where the protagonists are native to their respective fantasy settings. That's not even counting ones set in fantasy VRMMOs, which aren't exactly isekai either.

Isekai is also built more on escapism from current capitalist hell, this especially makes sense with many modern Isekai from Japan being centered on young men leaving behind a shitty life (salaryman lifestyle) that has no value (i.e. none of them are very concerned with returning home to their parents or other loved ones). This is distinct from female styled Isekais which arose in the 90s in which the protagonists would eventually return to their original world after having gone through a period of growth and development. This also kind of illustrates a very evident line between fantasy wish fulfillment for male anime consumers versus the very strict focus on female protags fulfilling their goals and returning as "responsible" contributers to society (i.e. Japanese traditionalist perspective on gender roles).

It's one of the differences between the Isekai and the Portal Fantasy. Sure sometimes the protagonist ends up staying, but the portal is in principle always there. Think Outlander where at this point they're popping back to the present for toilet breaks.

Heck sometimes in the more Isekai-like variants they start up a roaring cross-universe trade.

But ultimately that adds a sense of responsibility that Isekai lacks, because you never know when the FBI is gonna start wondering where all the gold and jewels are coming from your "technically not a smuggling operation because you technically don't cross a designated border."

At this point, it's just authors chasing the most successful current genre.