I was also a former retail worker, but I guess my experience differed slightly from yours.

My primary responsibility was not getting the carts. I had a lot of other things that I was responsible for throughout my working day that had to get done. But when we needed to get the carts, I would often be the one asked to go.

When people left the carts all over the place like animals, it made my job take longer. And that meant I had more to catch up on back in the store once I finished. Not to mention, being out in the hot sun, or the rain, or the snow was not my definition of a fun time, so I typically wanted to be done with it as quickly as possible.

People would just toss their carts into empty spaces or onto foliage dividers, even making eye contact with me and smiling as they proudly sauntered back to their soccer mom SUVs, as if they somehow believed they were giving me the gift of job security. Sometimes they also confused "shopping cart" for "trash can". I hated those people. I honestly wished death on some of them. I still can't stand people like that today and I make every effort to not be one of them now.

Years ago a buddy of mine worked at one of the big chain stores. He would always talk about how much he loved to be sent on cart retrieval, for the reasons you mention.

Interesting, I've always put my cart back with the thought that someone won't have to work as much if I do