Alternatively you turn it around and play it like a buddy cop movie: The straight-laced paladin screwed up and ended up breaking the law big time. He's given probation of he does some stuff for the city and the probation officer happens to be the loosest cannon in the entire force.

Cue the rogue constantly operating on "eh, it's probably going to be okay" (and getting his way because he's the actual cop) and the paladin burning half of his spell slots on curing the ulcers he's developing.