randomly reading different philosophical works won't give me much in-depth knowledge.

I'd rate reading different philosophical works much higher than reading modern summaries of philosophical movements.

I've rarely found that the generalization of an ancient philosopher brings much into my life, but often found that a specific turn of phrase or idea in their actual writing has offered a great deal.

As for recommendations, my top recommendation is Plato's The Apology which can be read in an afternoon and does a wonderful job in capturing the value of Socrates' commitment to rejection of false positives and negatives while also contextualizing the depth of that commitment against social pressures.

After that I'd recommend Leucretius's The Nature of Things mostly for the fact it's the only extant work from antiquity to have effectively proposed evolution nearly two millennia before Darwin - a detail that's frequently overlooked in the modern discussions of Epicureanism which tend to focus on its broader discussion of hedonism in the face of death.

I'd strongly recommend reading the philosophers themselves directly, sampling from each and continuing to read those that resonate with you, taking pearls of wisdom where you find them. And skip those you find insufferable to read.