It's partially but not entirely true. Having correct technique in your upper body matters too, as does accuracy, timing and the ability to create collisions.

All else being equal in terms of technical skill and leg strength, the guy with the bigger arms, fists as shoulders will have a stronger punch.

I had a ton of muscular atrophy in my right upper body due to a bulging disk in my neck --since corrected by surgery-- and I definitely noticed a huge diminution in my striking power, as did my regular sparring partners at the gym. So it definitely does matter.

I'm doing better now, but still not back to 100 percent and probably never will be. But that's OK since I'm pretty old anyway.

They explain the physics of it pretty well here, I think https://m.youtube.com/watch?v=LNfAwWk33nI&t=1m25s