If this happen for you in every battle- it's probably caused by your connection. If only in some of fights- might be opponents' fault.

I also suggest refreshing game just before entering SWB- for me, if I leave game running for a longer time it get's laggy and refresh fixes this issue.