Timing of when you do Matsuri also plays into factor. If you do it right in the beginning, you'll more likely to see more players equal or stronger since weaker players haven't had the time to be able to get wins so they won't get matched to stronger players (most likely you). Just like what PriaseLuka said, there needs to be enough people queuing in your level range that fit the criteria of "facing a lower/higher power player" and since most activity are from players level 80+ on middle-older servers. Of course a stronger and weaker player can be subjective if a player has a strong lineup vs a weak lineup regardless of their power as the system can only look at power and level to determine the matching.

If you still get frustrated with the system, just do it for the minimal rewards (battle three times)and those rewards add up over time.