For me mini client has no load times apart from enemies and cache is stored at AppData\Roaming\Brotsoft, as far as i know it never gets auto cleared, i have to delete it manually every time, even the mini client doctor thing does not work.

So try go there, in there will be folder called Player, just delete it and try again.