No, because agriculture isn't about minimising labor, it's about maximising the productivity of a given field. While you can sustain more people from a smaller territory, the process necessitates a division of labor where some have to make and fix the tools or tend to the livestock while others cook, till the land or collect and sow the seeds, etc.

It had very little to do with getting an easier life and more with preventing famine by way of ensuring a surplus in foodstuffs.

I suppose, but since there's a much more limited supply of gatherable food, there's an upper limit on the time you can spend, and the size of community it can support.

Agriculture doesn't have that upper limit (well, arable land limit but that's still much more), plus it takes a ton of work to sow crops, irrigate water, and wait months for harvest. Much harder than just picking berries for an hour or two a day, which is why the transition to agriculture took so long even after it was discovered.