To be fair, at least with a lot of games I play, the task is usually persistently displayed on screen and there’s either some type/combo of guided navigation, quest marker or glowing thing highlighting the thing you need to do/interact with.

Then again, I still get distracted with side quests/shiny things but much easier to get back on task when the information and optional guidance is right there