I think you're a victim of availability bias. You're thinking of all the women who sing songs about relationships and forgetting all the men who sing songs about relationships. You're remembering all the men who sing about not-relationships and forgetting all the women who do the same.
Tik Tok, Royals, I Love Rock and Roll, Girls Just Wanna Have Fun, Tom's Diner, All I Wanna Do, Anti-Hero, 9 to 5, the list can go on indefinitely.
And that goes doubly for comedians.
Mitch Hedberg did jokes about relationships. "I don't have a girlfriend, but I do know a woman who would be mad at me for saying that."
There are dozens of currently active women who do comedy that isn't just about relationships. Most probably include jokes about relationships, but comedy is about shared experiences, and relationships are fertile ground for material.