I never took it that way at least. Tom was developing a friendship with Kes which made Neelix jump to conclusions cause he's Neelix. Although someone did mention an alternate timeline episode where he marries Kes, I don't remember that episode cause voyager is forgettable af, but from my recollection he didn't seem to have any romantic or sexual intent and Star Trek tends to make that super obvious.