To help the narrative flow, I'm going to repeat a few points that you already know and which others have made.

One idea that might help with thinking through this is that 'communism' isn't a utopian end state of human society. Well, maybe it is, depending on what you mean.

Socialism is the next stage of human society because it is the only way to resolve the contradictions of capitalism. Currently, the logic of capital leads to a smaller and smaller ruling class and a bigger and ever more concentrated working class; it's not sustainable.

The ruling class can't exist without the working class but for the ruling class to exist, it must strip as much as possible from the working class. When this reaches a tipping point, the working class cannot reproduce itself. If there are no workers, the ruling class cannot sustain itself. If there are insufficient (worker) consumers to buy whatever commodities are produced, it becomes impossible to make a sufficient profit for the ruling class to sustain itself.

Capitalism also relies on infinite growth, which requires infinite resources and energy and an environment to absorb the infinite byproducts (waste) of production. But there are only so many resources and the environment is just about as full of waste (e.g. greenhouse gasses) as it can hold. These are just a few, albeit among the most antagonistic, of the contradictions of capitalism.

Only socialism can resolve these contradictions. Left unchecked, the capitalist ruling class will cannibalise itself in one way or another and destroy the ability of the workers to sustain and reproduce human social life. This is life under the dictatorship of the bourgeoisie.

Socialism, which means a dictatorship of the proletariat (DotP), resolves the contradiction by distributing wealth more efficiently. Rather than depriving the workers of the means of subsistence (real access to food, shelter, clothing, medicine, etc) to feed an insatiable capitalist ruling class, a DotP can distribute the fruits of production to those who need it, when they need it. Rather than producing more and more just for profit, we can say, no more of that, please, it will produce too many greenhouse gasses. Depending on how you look at it, China has either not yet reached socialism or is in the preliminary stages of socialism (it's wary of claiming too much).

If the working class, the proletariat, seizes and maintains power during the DotP, it becomes the ruling class. The contradiction between the capitalists and the workers is the final antagonistic class division. Once the proletariat becomes the ruling class, it means there is no longer a capitalist ruling class.

As the capitalist class and the proletariat are dialectically related, one cannot exist without the other. So at the final stage of socialism under the DotP, class distinctions will evaporate. Note that I'm being a bit reductionist here, there are other classes, but they'll confuse the picture and they're not at fundamental odds like the capitalists and workers; i.e. they can work together until their distinctions disappear.

A state is a tool for organising and exercising class power. It's necessary under capitalism and later, under socialism, to suppress first the workers, then the capitalists. But once there are no classes, there is no need for, and no logical possibility of a state.

If the workers are in charge and they ensure that every worker (which means everyone) has food, clothing, shelter, education, medicine, etc, there is not much need for money – what would you even buy when everything you need to live a good life is readily accessible and provided by you to others and by them to you?

This final stage of socialism is the eve of communism: classless, stateless, moneyless society.

As this reasoning is based on a dialectical materialist conception of history, there is an important caveat to make. For simplicity, we might speak of a neat chronology: first this happens, then that happens. In reality, the past, present, and future, and all processes within each period, are dialectically (internally) related.

This means that socialism has already ’started’ and that when we get to the eve of communism, it will already be underway. It also means that there will be traces of capitalism in human society right the way through and possibly into communism. Just as there are still many traces of feudalism in today’s capitalism.

To summarise:

• feudalism resolves the contradictions of primitive communism
• capitalism resolves the contradictions of feudalism
• socialism resolves the contradictions of capitalism
• communism resolves the contradictions of socialism
• something else resolves the contradictions of communism but, unlike in all earlier epochs, the contradictions of communism will not necessarily be antagonistic, which opens up a very different (again, practically unimaginable) range of possible futures.

During the later stages of socialism and into communism, ‘states’ may be unnecessary but there will still be forms of governance (see e.g. Roland Boer, Friedrich Engels and the Foundations of Socialist Governance for some material on this idea).

Pashukanis (a Soviet legal theorist) suggested that law would dissolve under socialism, arguing that most crime would be solved by improved living conditions, education, and medicine. Civil disputes would be resolved by something more like a train timetable. This makes more sense the more you think about it, considering how much law is arguments over property rights (even human rights litigation often comes down to whether a ‘human right’ violates someone’s property rights – like the right to a peaceful family life interrupted by an airport’s right to fly noisy planes in the dead of night – the airport and the country’s economic rights win in that situation, if you were wondering).

As for hierarchies, these may exist but they will be very different to the ones we know. This example obviously cuts both ways but in general, women become subservient to men because material conditions force them to rely on men for shelter, food, clothing, etc, especially if they have children. The hierarchy between men and women will properly begin to disappear when women have guaranteed housing and employment, and when childcare is free and widely available.

As for racial hierarchies, I accept the racial capitalism thesis, that race and racial hierarchies are entrenched in capitalism because it justifies the oppression of the majority of the planet for the wealth of the paler minority. Under socialism, the material basis of racism and of racial hierarchies will begin to disintegrate. The concept of race itself will disintegrate (there is certainly no biological foundation to keep it around). Going back to the point about dialectics, the dissolution of racial, gendered, sex-based, and other hierarchies will take a long time and it will be uneven.

As for technology and complexity, three points. First is that technological advancement is necessary for socialism and communism. China faced this problem in the inverse – starting with feudalism and a massive peasantry, it lacked complexity and technology. After improving the relations of production, their forces of production were lagging behind: people were more equal but the country couldn’t produce enough to lift everyone’s living standards.

Technological advancement was needed to make production more efficient. For example, if manual agriculture needs everyone in the village, who’s going to spend seven years in medical school or five years learning to teach, only to come back to the village and not farm? You need to develop the technology so that five farmers with tractors can do the job of 100 farmers, freeing up 95 people to do other things.

Second, as this example shows, socialism will make human society more complex than capitalist society because it is always looking to minimise unnecessary labour, to make life for workers easier, whereas capitalists only need to be slightly more efficient than their competitor (until they have a monopoly, when they can be as inefficient as they like).

(At the same time, this process will also simplify a lot of things by cutting out the middlemen and the red tape. All the needless complexity due to intellectual property law (IP), for example, will disappear: not only will nobody need brokers, permissions, lawyers, and all the related institutions; nobody will have to use deficient, overly-complicated tech because the IP for the simple tech is only available to licence holders or because IP leads to incompatible duplications.)

Third, the capitalist development of technology exacerbates the contradictions of capital. This takes us back to where we started. Increasing automation means fewer workers, but fewer workers means fewer consumers, which means fewer sales, which means less profit. The capitalists’ implementation of technology (to beat the competition) inevitably undermines the material base of their power: the ability to extract wealth from workers. The more complex and advanced society becomes, the sharper the contradictions, and the more that socialism/communism are needed to resolve those contradictions.

All this is to say that the transition to communism will take centuries (we’re already 150+ years at least into this transition) and it will happen because the contradictions of human society must be resolved. But as everything is contradiction according to dialectical materialism, new contradictions will arise once the old ones are resolved, which means that even communism cannot be a state of stasis; it, too, must change.

For further reading, you may enjoy The German Ideology.