Spamming ninja fragments every day is why you're not getting much for your stamina. The more you farm, the less they drop.

To reset them back to high drop rates you have to stop farming them.

What players used to do was farm ninjas for a few days then switch and farm gear. Then when they went back to farming ninjas the drop rates were better. This is the more efficient way to spend stamina, by spreading it around instead of focusing on a single thing.