You're not incorrect, and even "he was a product of his time" isn't an excuse: when he was alive, even other racists thought that Lovecraft was a bit too racist.

However, at the same time - you have to look at what impact reading his work has.

He's dead. He doesn't get money from it. The works are public domain. His estate doesn't get money from it. Further, the language used is striking, influencing a century of other work.

Does that language come from a place of racism? Yes. But it the work itself isn't overly racist - or at least, it doesn't make it more racist than Sherlock Holmes. Conan Doyle's The Sign of the Four is used in college classes today to teach Orientalism, yet largely people accept such a thing as okay because it doesn't radicalize new people into the subject.

If you reject every artistic work because the creators had questionable views, then you begin forcing yourself into strange choices. If the artist doesn't gain benefit from you reading it - then logically, it doesn't matter if you read something they made or not (contrast this to Harry Potter, where consuming said media gives money to a TERF). When the artist is out of the picture, the only thing that matters is what the work means to you.

You have the right to say "the work is abhorrent because of XYZ", but said things should be things you can point to within the work itself. If the artist isn't gaining benefit and their views aren't the focus of the work - why does it matter?