That fails to consider that if people speak of "an invasive species," they're almost certainly using the botanical definition of that word, which boils down to "a species not native to the ecosystem where the speaker as found it," and where native species are species that exist within an ecosystem without ever having had humans show up in said ecosystem.

Just because you like that plant being there does not mean said plant stops being invasive.

And if an organism has existed in an ecosystem for millennia, long before any human ever set foot there, then it doesn't matter how annoying it is, it is a native species.