Why didnt you just click the retreat *on instead of telling him that you will let him win? It also happens to me, and if I win him first time, I let him win the 2nd time. But if I meet him again 3rd time, I compete again, because like me, he is just trying his line up and not doing arena rewards.