Ehhh I don't really care about this. Landlords are categorically exploitative leeches, so this pales in comparison to the day-to-day activities of the landlord.

Besides, he should remove the plants whenever it was convenient for him. I'm not going to dictate the "should'ves" in this situation. They're his damn plants.

My husband had plants at the last house we rented, and we basically had to wait until the weather was right and we were just about ready to relocate before he removed and replanted them.