Sadly, the game mechanics are made in a * way.
Why even if you lost your previous match you can be matched with somebody way stronger than you even if you weren't waiting for more than 60 seconds?
There are 3 possible answers.
1) Your enemy was at his 1st fight and the 1st fight is random.
2) Your enemy was waiting since more than 60 seconds.
3) Your enemy was defeated too in his previous fight.
So, what you are thinking now is: how is possible?
Simple.
Is possible because the algorithm that manage the matching system doesn't take in account both the queues, but just the first one that was started.
So if your enemy was in one of the 3 situations i talked about and is in queue and you click on match, is his queue that gets priority over yours because he moved before you.
This can be easily solved by adding the condition that two defeated people can't be matched unless both are waiting since more than 60 seconds, but, hey, too much work to do here! And for what? To increase queue times? No way! Better leave this * as is now! (That's what they think).

Cross server space time, matsuri and swb are based on the same server bracket pull, i can confirm it for you due to the mess people in s295-306 were involved when they got merged with s291