Heat by itself is chaotic. It is the brownian motion of molecules vibrating. If you're in a hot room, using that vibration to get something useful done can be difficult. Try to think of a way to get those randomly vibrating molecules to do some form of physical work.

When we talk about heat gradients we mean there's one side that's hot I.e. moving a lot, one side that's cold, not moving at all. One of the laws of thermodynamics is things tend to stabilize, so the hot side and the cold side are going to mix they're going to want to move into each other and find an equilibrium. You can use that preference for movement to do mechanical work, like turning a paddle.

This is all broad strokes. You're trying to get organized energy, out of a chaotic system. Classically when humans use heat to generate workable energy they use the heat to make steam from water, and then they use the water to do mechanical motion.

None of these systems are 100% efficient. There's always going to be waste heat. Energy you can't capture. And that's the problem. It's the waste energy that's making air conditioners most inefficient.