And there's literally only one car to keep in mind, so it's way less problematic than a traditional 4-way intersection.

They didn't say they would yield, just take slightly more time to make the appropriate checks. And not sure about your car, but most have lights that indicate what they're doing (slowing/stopping and signaling); which a cautious driver would likely use more effectively than I would assume someone who just drives how they think other people are expecting them to would.