are only resolved by supply

Eh, I don't agree. Supply is super important, and we definitely need to see a lot more of it. We need a doubling or tripling of the amount of government housing. We need zoning regulations to allow medium density at least in all cases. We need higher density in inner suburbs and near train stations.

But regulation protecting tenants is also useful. Vienna is probably the gold standard, with multiple different tiers of pricing regulation for different kinds of property and different needs. Other kinds of non-price regulation, like bans on no-grounds evictions, protections for tenants on quality and repairs, and rights to make minor modifications, are also important regulation. These go very well hand-in-hand with a levy like OP proposes, because the levy stops an owner from throwing up their hands and saying "I don't want the tenant to have these rights, so I just won't bother renting it out."

There's also just the simple fact that even with the very best of intentions, increasing supply is a years-long endeavour. Price controls can be done right now and take near-immediate effect. In a crisis, we can't afford to wait around for the long-term solution. We need something that works right now, in addition to addressing the root cause.