I'm of the opinion that if something is distracting a student, they shouldn't use it in class (without a very good reason). Which means if a student brings their phone into class, they better make sure it won't distract them. If they play with it when they shouldn't be or it rings, by all means, punish them just like you would punish talking in class.

But stuff like using it right up until the teacher actually starts teaching? That's not a problem. Or if it rings for a legit emergency (do not disturb mode can allow this), that's totally fine. If some assignment actually benefits from a phone, great! If you finished an assignment early, go ahead and use it so long as you aren't disruptive.