Tldr leather can be a component, but not the whole thing

So basically when sharpening, you have a few different things that all do slightly different jobs.

A sharpening stone will remove a little metal along the edge, very slowly, but that makes the edge roll over slightly along the opposing edge face. This makes the edge slightly jagged, you can usually see the reflection of light along the edge of a knife to see it. You don't want to see any shiny reflection on the edge.

To remove the rolled over edge, one method that is quite popular is to take a leather strip, rub some polishing compound into it, and you run the edge of the knife along it in a similar fashion to how you use a sharpening stone. The softer surface combined with a milder abrasive does a nice job, if you do it right.

The end result is a razor sharp knife.