but you don't need cargo all day, stores generally restock in the off hours or maybe just straight up at night. Certainly grocery stores restock the actual shelves from their internal storage every morning.
So you'd do a cargo tram run every night when there's plenty of time to do things slow and not have to worry about passenger services.

As for switches, that's the thing! you wouldn't need extra switches since you can operate outside of passenger hours and just use the existing platforms most of the time, maybe building a little extra platform where needed.

The only really new part of this is using trams, it's already perfectly normal for most deliveries to be made in the off hours in places where there's a lot of pedestrianization.