You keep your skill levels (if same lvl group or above), but any group points you have will be lost. So spend before leaving if you have a lot.