Yeah you're almost always going to increase loading time and for most people who don't play the mini game it'll just be annoying. Most people look at their phone if a loading screen is longer than a few seconds anyway or have a drink or something.

I think the main reason is that you're trying to dump and reload game assets quickly, it's far easier to do that without also doing something else