Yeah, Bermuda grass is the devil, is extremely hard to get rid of, and will out-compete clover and most other ground-covers. You're probably always going to have problems with it creeping in no matter what you do. It won't out-compete shade-loving plants in shade though, and can't out-compete taller plants.

If you don't mind chemicals, I've used a grass-selective herbicide (fluazifop or clethodim can't remember which) to kill Bermuda grass in a rock bed and it worked ok (took many applications).

I've also suppressed it by sheet mulching with multiple layers of cardboard and 4"+ of woodchips on top. Still required some weeding after a while.

I've never tried it, but I've seen people kill grass by covering it with a dark landscape fabric. However, I think this would take many months to kill Bermuda grass.

Also, I'm not sure clover thrives where Bermuda grass does. Where I live, clover dies in the summer (too hot), and doesn't like full sun in spring and fall.

Personally, I'm slowly replacing parts of my lawn by adding and expanding beds (sheet-mulch + woodchips), and planting fruit-trees and perennials in them. This seems more manageable to me than trying to do my entire lawn at once (I have a fairly large lot). I walk around the beds about once a week and pull any weeds I see, and pull weeds in beds I routinely walk by whenever I see them.