This might sound like a question inspired by current events, but I've actually been thinking of this for a while and can give pointers to a few times I had asked this or talked about it.
The people who the masses look up to seem to have a strange way of dishing out their opinions/approval/disapproval of the groups of the world. Some groups can get away with being considered good no matter how negative their actions are while other groups are stuck with a high disapproval rating no matter how much good they might do, and a discussion on whether "culture" or a "cult" is involved almost always comes up.
An example of this is the relationship between Islam and Scientology, in fact this is the most infamous one I can link to having spoken about. People on a certain side of the thinktank spectrum (the same side Lemmy seems to lean towards at times) are quick to criticize Scientology even though they consider "classic Islamic philosophy", for a lack of a better way to put it without generalizing, as not inspiring a call for critique to see how one may change it. And I've always wondered, why? One at times leads people to trying to exterminate innocent groups, the other one is just "Space Gnosticism" that has a few toxic aspects but hasn't actually eliminated anyone. Of course, I'm not defending either one, but certainly I'd rather live in a stressful environment than one that actively targets me.
This question has been asked a few times, sometimes without me but sometimes when I'm around to be involved, and they always say (and it's in my dumb voice that I quote them) "well Scientology is a cult, of course we can criticize them" and then a bit about how whatever other thing is being talked about is a part of culture. This doesn't sit well with my way of thinking. I was taught to judge people by the content of their character, in other words their virtues, so in my mind, a good X is better than a bad Y, in this case a good cult should be better than a good culture, right? Right?
In fact, as what many might call a mild misanthrope, I'd flip it around and point out how, over the course of human history, alongside seemingly objectively questionable quirks people just brush off (like Japan for a while has been genociding dolphins for their meat value just above extinction "because it's culture" or how there are people in China who still think dinosaur bones are a form of medicine waiting to be ground up), no group/culture has kept their innocence intact, every country having had genocides or unnecessary wars or something of the like, things they ALLOW to happen by design. Then they turn around and tell so-called "cults", even the ones that have their priorities on straight compared to cultures, that they are pariahs and shouldn't count on thriving, even though their status is one that doesn't necessitate gaining any kind of guilt. I was a pariah growing up, almost everyone else revolved around a select few people that seemed in-tune to the culture, and they would say anyone who revolved around people outside the group (me for example) was "following a cult", and this hurt at the time, but now seeing all the wars going on right now, I might consider this a compliment.
My question, even though it by definition might make affirming answerers question whether they are pariahs or a part of the cultural arena, is why does nobody agree? Why are cultures "always precious" while cults are "always suspicious"?
As others have said, there’s never going to be a clear cut line between the two. I think it’s more useful to take a functional perspective. Something isn’t problematic because it’s a cult; it’s a cult because it’s problematic. I like Hassan’s BITE model of authoritarian control here. We look for social systems that are purposefully organized to enforce different kinds of control over individuals within the system - Behavioral, Information, Thought, and Information control in the BITE model. We see where systems rely on mechanisms of control to the clear detriment of those within the system.
You mention in another comment the idea that many “cults” are going to be relatively more accepting of you than many “cultures.” That’s undoubtedly true. But the distinction is in what happens next. The border around a cult system is only permeable in one direction. You may be accepted with open arms, but that acceptance is a tool to get you into a place where you can’t leave because you won’t (or feel like you won’t) ever be accepted again outside the cult.
The control mechanisms also create an all-in system. I’m not generally a fan of religion TBH, but you can decide how much you want the culture of Christianity, Judaism, Buddhism, or whatever to affect your life day to day and in what ways. If you’re in a Christian cult though, like the IFB or IBLP (the one the Duggars are in), the system decides your level of involvement. Scientology is a great example of this because it looks like there is a wide range of involvement level. You see a lot of celebrities who don’t seems crazy, who talk about how wholesome it is, who say they’ve never seen any of the abuses people talk about. It’s not that these celebrities are opting for a chiller version of Scientology, it’s that Scientology opted them into a less obviously, outwardly repressive day-to-day for the benefit of the system.
All this to come back to my first point - this is a functional distinction, not a formal/semantic one. Is some social system manipulating its members in an organized and harmful way? Then let’s call it a cult so we can talk about that concept more easily. THEN the question of is this or that group a cult based on whether it functionally presents as one.