It's playful banter. The interviewer starts it with "you can't do that to Black Media" jokingly when he needs to leave, and his response is pointing out the obvious. The kind of Black people consuming Black Media aren't going to be voting for Trump, so it'll be ok that he has to leave. Biden isn't a great public speaker. He didn't word this one well. It wasn't a prepared statement, it was a response to other playfuyl banter. It's primarily been used against him by right wing media in false outrage.

I can't be bothered to care about playful banter. He's not redirecting government resources or ignoring folks that didn't vote for him like the last administration. The context above makes it clear what he meant too.