This decision is all about you.

I'm an introvert who works with people, I could be a recluse all year and I'd be happy. Without work maybe I'd be a little lonely at times, but there is ways to fix that for me, without relying on neighbours.

You seem to like having neighbours though, so that's very different. If that is something that worked well for you in the past, I think that's an indicator for the more expensive house. It's a permanent thing, after all, and if you're rather extroverted or at least need humans around on occasion, then you shouldn't make yourself unhappy by buying cheap.