Someone already mentioned Craig so I'm going over the rest.

• Brosnan was 48? during his last Bond movie and I think he looked good in The November Man, which was 10 years after his last Bond movie. He's 70 now so obviously he looks like shit now.
• Dalton looked great in Hot Fuzz and that was almost 20 years after his Bond movies. Dalton is nearing 80 so he also obviously looks like shit now.
• Moore, if I did my math right, was almost 60 when he Bond career ended. I have no idea what he did after Bond but when you stop being Bond at his age it shouldn't come as a surprise if you physically fall off.
• And then we have Connery who had a long career after Bond.

I'd say it's been mostly Craig who has "fallen apart" but I also think he bulked up for Bond and it took a toll because he pushed beyond his natural physique (he was much leaner before Bond).