Well, you're causing someone to lose out on some of their reward, for your own personal benefit.

I don't plunder, and I don't care too much if I get plundered, because I know it's a part of the game. But it shouldn't be hard to see why people would think you're rude.

I'm not saying that talking about you to others is justified. Just saying why they're doing it.