Even if you’re passing people, you’re probably still obligated to move to the right, by law. You’ll have to check your state laws for your specific obligations.

If somebody is riding your ass or going around you to the right, they absolutely should not be doing that. However it also means you’ve likely been camping in the left lane, impeding the flow of traffic for far too long by the time you reach that point, and you’re considered part of the problem if you’re causing other drivers to pass you on the right to get around you. When you’re driving on public roadways, it’s not about you. You need to do your part to ensure a smooth flow of traffic, so that everybody can arrive at their destination safely.

If you want to drive slowly, just do so in the rightmost lane. They literally have a lane just for you to do that, where you can be free of tail riders, and it’s impossible to be passed on the right.