Refines come second to slot/positioning, if I remember correctly.
You can have topped up, maxed level refining. . but still not go first if you're in like position 4 while the enemy is in position 1 with no refines. . .

Or am I wrong? But I'm pretty sure that that's how it works. . .
Maybe you were laggy or something? I dunno