Does anyone else on your SERVER experience this problem? If so, is it at least somewhat common?

If the answer to either question is "no", it's probably YOUR problem, I'd bet on packet loss due to unstable connection.

If possible, attempt a hard line connection (ethernet etc), because wireless can be unreliable in a number of ways(positioning of router vs your device, router, your wireless chip etc)
If the hard connection still have issue, it's either your modem or your ISP(internet service provider). You can probably get another modem easily enough to figure out which.