If you remove more than one ninja from formation, then put them back in, this will happen.
But if you replace ninja on formation with another ninja, the equipment will stay with the replacement (thus when you replace it back, it will return back to where it was)